It’s unclear when or why the words tussle and tousle divided into different meanings. A tussle is a physical struggle that suggests shoving and pulling. When you mess up someone’s hair, the word is tousle. If you’ve tussled with police breaking up a language debate that got out of hand, your hair might end up tousled. The words are close cousins, but for some reason we have two spellings for similar concepts.
We’ve been tussling since at least the 16th century. The noun form of tussle came later. Tousled hair might have come earlier, but we’re probably talking about spelling variations on the same word. Both words are from touse, which means pull.
Samuel Johnson’s 1755 dictionary has an entry for to touse (to pull, to tear, to haul, to drag), but nothing for tousle or tussle. Webster’s 1828 dictionary has all three, using the spelling tousel to mean to put into disorder or tangle (“used by the common people of New England”).
Most dictionaries today mention hair in their definitions of tousle. If pushing and shoving are involved, the agreed-upon word is tussle.
Unrelated is tussal, which relates to a cough, which is tussin in Latin. Hence the popular cough remedy Robitussin.