[support] Impossible to delete user picture

I am having an odd problem with user pictures.

I uploaded one, let's call it Picture A.

I then deleted it and my user pic disappeared. As expected.

I then tried to upload another jpg image and got the message "the uploaded image was not an image". So then I tried another and Picture A appeared again.

So I went to files/pictures and manually deleted it and ran through the previous two stages again with exactly the same results. One jpg got rejected as not being an image and magically, Picture A reappeared again.

I am using simply Upload module and have never seen such malfunctioning with core modules. Has anyone seen anything similar before?

Neil