The blind-folded classification dataset

This is description of our blind-folded classification dataset which was used to test generalization of the object identification neural network in our paper Learning the signatures of the human grasp using a scalable tactile glove trained in visually aware conditions to conditions without visual awarness of the grasped object. See "Methods » Dataset acquisition methods » Visually aware and blindfolded conditions in the paper".

The dataset is provided for non-commercial use only. Contact us with inquiries about commercial use. Read the license for more details.

Dataset content

The zip file of the dataset contains following structure:

|
|- metadata.mat						
|- [batch]
|     |- [recording]
|          |- [000000...N].jpg
|          |- viz
|          |    |- [000000...N].jpg
|          |- pressuredata.mat

Where:

metadata.mat is the main data file with all the information needed for the experiemnts in our paper. Details bellow.
[batch] stands for the name of the batch folder that contains multiple recordings from the same recording session, e.g. "001"
[recording] stands for the name of the recording that contains a continous sequence of frames, e.g. "003-tea_box"
[000000...N].jpg are video frames corresponding to the recorded pressure maps (not used in our paper), e.g. "000452.jpg"
viz/[000000...N].jpg are downsized pairs of video images and corresponding pressure map visualizations. The pressure is visualized without spatial layout. The values are mapped from range 500 - 650 to range 0 - 1 in logarithmic space. The green bar bellow denotes validity of the frame for our neural network as described in the dataset preprocessing in the paper and corresponds to the flag "hasValidLabel".
pressuredata.mat is the raw recorded pressure data available for every recording. Contains timestamp tStamp, corresponding frame number and the pressure values data as int16 array of size Nx32x32 where N is the frame count. The pressure values range between 500 and 1024.

Metadata structure

The metadata.mat contains all the pressure values for all recordings as well as all the flags generated in the preprocessing step to filter valid frames. More details in the Methods section of our paper. The file is a matlab MAT-file. Note, that it follows C (zero-based) indexing (unlike Matlab). This means that in Matlab you have to access dictionaries with 1 additon, e.g. batches{batchId(55) + 1}.

The file contains:

batches - list of batch folder names, e.g. "001"
batchId - per frame index reference to batches - zero-based
recordings - list of recording folder names, e.g. "003-tea_box"
recordingId - per frame index reference to recordings - zero-based
objects - list of object names, e.g. "multimeter"
objectId - per frame index reference to objects - zero-based
frame - zero-based frame index corresponding to video image file names
ts - relative frame timestamp in seconds since the first frame
pressure - the pressure frame as a sequence of 32x32 int16 values with range between 500 and 1024. Note only 548 pixels have a valid value corresponding to a physical sensor on the glove.

splits - definition of splits used in our experiment ("train" and "test")
splitId - per frame index reference to splits (zero-based) defines if frame was for testing (1) or training (0)
hasValidLabel - flag marking that the object is being held based on the threshold procedure described in "Methods/Object identification"
isBalanced - flag marking that the frame was used in the set subsampled to achieve uniform frame count per object class as described in "Methods/Object identification"

actionStartTS - timestamp marking start of unaware object manipulation
actionEndTS - timestamp marking end of unaware object manipulation, i.e. moment when the user recognized the object

This information is also useful to associate the video image with respective pressure frame. To read an image associated with frames stored on index 123 in Matlab do:

load('metadata.mat');
im = imread(fullfile(batches{batchId(123) + 1}, recordings{recordingId(123) + 1}, sprintf('%06d.jpg', frame(123))));
fprintf('Showing object "%s".\n', objects{objectId(123) + 1});
imshow(im);