Via Infocult, some links to early experimental Henson films that are available online:
Time Piece, a 1965 surrealist meditation on the fluid nature of time. The paper cut-out fireworks also showed up in several shorts that were on Sesame Street when I was a kid. Also, the credits list "Frank Oznowicz," which is the only time I’ve ever seen his full name used.
Organized Mind, a short that aired on the Tonight Show in 1974, is more psychedelic. The update on that entry notes that while the film is actually from 1967, Henson performs the mouth and eyes live.