The use of 'herb' as a popular insult in East Coast slang is probably the sum of a couple histories.

The earliest example I could find for 'herb' as a potential insult is from 1663, and has to do with the word's natural associations with youth, unripeness, and unreadiness:

"Since our haruest is but in the hearb." (E. Sparke, Scintillula Altaris)

From this point until the modern era, if herb was used as an insult, it probably followed from these types of metaphors. While it's a weak association, it gives an insight into the broader meanings that 'herb' comes to take in the 20th century.

By 1925, the name Herbert (etymologically unrelated to 'herb') was already thought of as a generic name for an odd, generic person. A dictionary of sailor slang written that year defines 'Herbert' as:

"A funny fellow... Also used colloquially in addressing any man whose name was unknown... From the typically cockney pronunciation of the name Herbert" (E. Fraser & J. Gibbons, Soldier & Sailor Words).

In the year 1929, we also see a sharp decline in the popularity of the name Herbert in America. This is almost certainly due to the huge unpopularity of President Herbert Hoover, who presided during the devastating stock market crash that year. By the year 1960, a 'Herbert' is solidly a complete idiot who ruins the party. For instance:

"At large jazz festivals the incursion of a minority of moronic roughnecks (known in the business as Herberts) is ruining the pleasure of the great majority." (Sunday Times, 11 Sept. p. 37).

The pejorative sense of 'Herbert' is then swiftly crystallized and mass distributed in television, most notably through Burger King's '85-86 "Where's Herb" campaign, in which a fictionalized nerd named Herb is repeatedly chastised for being the only person in America not to have eaten at Burger King.

The ad lost a ton of money for BK, but not before briefly dominating the late-80s TV scene (including an SNL cold-open parody) and making our critical switch from 'Herbert' to 'Herb'. The character himself is referenced on Public Enemy's single "Don't Believe the Hype", released in 1988:

"Word to Herb, yo if you can't swing this

Learn the words, you might sing this"

Given Public Enemy's massive influence along with the mass awareness of the ad, it's unsurprising that 'herb' is then quickly adopted as an insult by other hip-hop artists (including on major tracks by Biggie, Nas, and 2pac ). In this process, 'herb' is generalized, no longer a direct reference to the BK ad, and takes on some of the earlier botanical shades of meaning (slightness, weakness, tenderness, youth). 'Herb' as an insult is probably used most repetitively and extensively on the track "Herb" by 7L and Esoteric, off the album Dangerous Connection released in 2002:

You're a herb, if you say you dig all the time

But can't find a sample from beats to the rhyme

A herb, if you claim your raps is official

But everything you sample is a fucking re-issue

This is perhaps one of the early 2000s albums Dave refers to.

So there ya you go, ya herbs.

(Big thank yous to the OED, Oxford Dictionary of Slang, behindthename.com, the rapgenius search function, lifeasastrawberry.com for information and pictures, and to the twinnovation nation of course)