The worry hat is a grey/blue cap (probably knit wool) with earflaps and a ball of fluff on top which Marten gives to a worry-ridden Hannelore to counter her "nonspecific feeling of dread." Marten insists that the hat is made from "a special fabric that repels vagrant anxiety" and insists that after Hannelore puts it on, she announce, "Worry hat, engage!"
Hannelore is surprised to find the hat working, and Marten tells her she may keep it. Marten explains that the idea comes from his mother doing the same thing for him as a child, though it is unknown if Ms. Reed made the hat herself or bought it. The worry hat appears to be effective for Hannelore, supposedly giving her +10 bravery and +5 charisma, with the likely additional effect of bringing her emotional armor class down into negative numbers.
Later, Hannelore is wearing the hat while cleaning when she hits her head on an open cabinet door, getting a large bloodstain on the hat. She runs to Coffee of Doom, fearful that she's ruined the hat. Faye instructs her to clean it with hydrogen peroxide (while Dora is surprised that there's something which Hannelore doesn't know how to clean).
Later still, Hannelore comes by the apartment to show Marten and Dora that the worry hat came out fine in the wash. In spite of its effectiveness, the hat is rarely seen in the strip after these events, though Hannelore is seen wearing it while getting over the flu and when Hannelore joins in the sledding game of "speed beer" (both, naturally, being events which might cause worry). Later, Hannelore builds a card house wearing the worry hat.
The worry hat features again after Marten and Dora's breakup; Hannelore brings it to Marten at Smif Library to assuage his likely sadness (though, she admits, that is "not the same as worrying"); he dons the hat and she reminds him to recite the "activation phrase," and then snickers over how silly he looks in it, a side effect he notes, along with "headsweats, impaired hearing, and compulsive fiddling with the pom-pom."
A free pattern for knitting one's own worry hat can be found here: https://www.ravelry.com/patterns/library/worry-hat