Definition

A siamese neural network uses the same weights while working in tandem on two different input vectors to compute comparable output vectors. Usually these two output vectors are then compared using distance metrics.

They are used in:

Examples

Signature Forgery Detection, Presentation Slides

Siamese network

![[SiameseNetworkSignatures.pdf]]

Allows us to send two signatures through two identical networks. One being a real signature, and the other one where we wish to know if it was forged or not.

Deszö and Dominik have recreated it here: https://github.com/DKormann/check_sigs/tree/main
The original repository is here: https://github.com/AtharvaKalsekar/SigNet?tab=readme-ov-file

Ignore the old repo: https://github.com/MrGilga/signature_app

Distance based approach

General idea

Idea: We have more information than simply two signatures. We have quite a bunch of signatures from one person.

Outlier detection

Instead of training a neural network to detect the fake signatures, we could try unsupervised clustering (of each person) and detect the main cluster. The outliers would then be flagged as forgeries. This is basically an outlier detection problem. Most clustering algorithms expect multiple clusters, and might force the data into many different clusters. Distance based approaches are better if we only look at a single person.

I see no reason to look at multiple people at the same time. Let's detect outliers per person.
Nes knows a lot about that
Let's look at isolation forest as well it is a similar (not the same approach) and creates nicer graphs

Dataset

Dataset: Cedar: https://www.kaggle.com/datasets/shreelakshmigp/cedardataset/code
55 people created 24 real and 24 forged signatures. So we will have 1320 real signatures and 1320 forged ones in total.

this differs from what we will get from the ministery. We will basically have 1320 real signatures, and a few false ones.

We can expect there to be mostly real, and a few forged signatures. -> dataset with around 24 real signatures per person, and 5 forgeries per person.

I therefore created a dataloader which gives me the real signatures and n (initialised to 5) random forged signatures per person.

Data preprocessing:

There are a few issues with the cedar dataset we need to fix. The background for real images is much more white than the one for the forgeries. We need to preprocess the data, by "binarising it".

Embedding

I want to embed images. I don't have a big dataset, but maybe I can use another model, that was used for handwriting signature and use those embedding vectors. I have used Vgg16 and it kinda works okay. It is also mentioned here: https://blog.fastforwardlabs.com/2021/05/27/pretrained-models-as-a-strong-baseline-for-automatic-signature-verification.html where they used the second to last layer as the embedding vector.

As described in this blog post, let's use the second to last layer from VGG16. For best performance we will preprocess the data (after binarization) the same way VGG16 did their training data.

  1. Input size: 224, 224
  2. We want RGB images again.

Perhaps there are better ways to do this. Like a model specifically trained on signatures. This is what Dominik did. https://github.com/AtharvaKalsekar/SigNet/tree/master

Maybe I can do the same, using their model.

We would need to split the testing dataset before training this embedding model.

Distance based method

I choose to use euclidian distance instead of cosine similarity. It is simpler, and I don't expect any major differences in "magnitude", the size of the signature or the intensity. If there was a major magnitude difference, I would even expect this to be an indication of forgery and not something I would want to ignore in the distance measure.

At the end I might want to do an ensemble method with both metrics.