Modern detector manufacturing allows spectral and polarimetric filters to be directly integrated on top of separate detector pixels. This enables the creation of CubeSat-sized spectro-polarimetric instruments that are not much larger than the detector and a lens. Redundancy inherent to the observed scene, offers the opportunity for sparse sampling in the form of not scanning all filters at every location. However, when there are fewer pushbroom steps than filters, data are missing in the resulting data cube. The missing, largely redundant data can be filled in with interpolation methods, often called demosaicers. The choice of filters and their precise layout influences the performance of the instrument after the demosaicing process. In these proceedings we describe a part of a design toolbox for both the filter layout and the optimum parameters for the reconstruction to a full spectropolarimetric data cube. The design tool is based on training a (neural) network and jointly updating the values of the filters and demosaicer. We optimized a filter layout by training on spectro-polarimetric remote observations of the Earth acquired by SPEX airborne. This optimised filter layout could reconstruct a validation scene from five overlapping snapshots (pushbroom steps), which would take 109 pushbroom steps when measuring with a classical layout and no reconstruction.