I usually try to remove the bark.
The bark and the cambium layer on the outside of the wood contain sugars that will make the tase bitter.
Heartwood is the stuff that has the taste you are seeking, also green wood should never be used.
Wood that is cut in the spring and early summer will be easier to strip of the bark than wood that is cut in the winter when the sap has retreated to the roots, and the outer layer of cellulose has started to harden into heartwood.
A little bark won't hurt though.
I bought a small wood chipper from DR Power and chip up hickory and fruit wood. The wheelbarrow is filled with apple wood from a dead tree on my property. I usually try to pick out most of the bark fragments when I bag it.
Mostly I like hickory that has been laying on the ground for a year or so and has a little spalting in it. It imparts a sweet savory taste to the meat that reminds me of the taste I remember when I was a kid and eating barbecue in south central Virginia.
It is that taste I have been hunting for all my life.