An Exploration of Mannequin-State Information in Anomaly Detection | by Sara Nóbrega

The Information

MNIST Pixel Information

The primary dataset employed right here is the standard MNIST pixel information, comprised by hand-written numbers. Right here, the background is black and the digits are white.

Anomalous MNIST Pixel Information

To check the brand new process and examine it to the standard one, I created 4 easy varieties of anomalous information.

The aim was to check every technique’s detection capabilities throughout a small spectrum of noise variations, incrementally intensified from one anomaly kind to the subsequent.

The noise fee will increase from the primary to the fourth kind of anomalous information. As you may see within the determine beneath, within the first and second sorts of information, the noise isn’t even detectable to the bare eye, whereas within the third kind, you may already spot some white pixels.

Determine 2: 4 sorts of anomalous information | Picture by writer

Mannequin-state information

Whereas MNIST pixel information, with its hand-written digits in opposition to a stark backdrop, gives a basic basis for anomaly detection, we’re attempting one thing else. It’s a little bit of a leap, taking us proper into the core of the educated ANN to see what the neurons are as much as. This might give us a complete unique approach on recognizing anomalies.

As talked about, this mannequin state information is comprised by the state of the neurons in an ANN when educated with MNIST information. As such, to generate this information, we begin with coaching a easy ANN with MNIST pixel information, each with regular and anomalous information (the anomalous are comprised by the noisy information confirmed earlier than in Determine 2).

We then carry out the standard: cut up the info into coaching and testing, after which we match the ANN mannequin:

mannequin.match(X_train,Y_cat_train,epochs=8, validation_data=(X_test, y_cat_test))

After that, we wish to retrieve the names of the layers in mannequin and retailer them in a listing:

record(map(lambda x: x.title, mannequin.layers))

Lastly, we create a brand new mannequin that takes the identical enter as the unique mannequin however produces output from a selected layer known as “dense”:

intermediate_layer_model=Mannequin(inputs=mannequin.enter, outputs=mannequin.get_layer("dense").output)

That is helpful for extracting info from intermediate layers of a neural community.

Let’s check out this model-state information:

model_state_data_layer1=pd.read_csv("first_layer_dense.csv",header=None)
model_state_data_layer2=pd.read_csv("second_layer_dense.csv",header=None)model_state_data_layer1.head(4)

Determine 3: Snapshot of Mannequin-state information (First layer). | Picture by writer

The model-state information of the primary neural layer is comprised by 32 columns and 4 rows.

With just some strains of code, we’re in a position to extract information from the intermediate layers of a neural community.

To check the effectiveness of the brand new technique, I’ll be utilizing information from each the first and second layers of the neural community.