First observations of the full Stokes vector in the upper chromosphere are presented. The He I 10830 ̊A line, which has been shown to give reliable measurements of the line-of-sight component of the magnetic field vector, has been used for this purpose. It is shown that the difference between the appearance of chromospheric and photospheric magnetic structures observed close to the solar limb is largely due to the difference in height to which they refer and projection effects. The observations do suggest, however, that the magnetic field above sunspot penumbrae is somewhat more vertical in the chromosphere than in the photosphere.