There are often worms inside the husks that are larvae of the husk fly. These insects seldom damage the nut inside the hard shell so you should not be concerned.
Remember that the black walnut produces a substance that is toxic or "allelopathic" to other plants and is called juglone. Tomatoes and coniferous trees are especially sensitive so take care with husk and seed disposal and do not put in compost. This mild toxin helps the tree keep other vegetation from competing or valuable nutrients and moisture.
