Can You please upload the link to the data set(MANN) ?

[shamanez]

I assumed you have used this data set. But it has only 964 classes.

But you have used a number of training and testing classes respectively 1200 and 423.

Also when training my training loss seems to keep around 80 , for about 2000 epochs.

[snowkylin]

That is only part of the data set, another part is images_evaluation.zip which contains 679 classes. You can add the two part to get the whole data set and divide it by yourself. I do not remember exactly whether I added the evaluation part to the whole data set. If not, the result might need to be recalculated.