To delete files, use the unlink function. Your upload isn't secure though. Read these:
http://www.tidy-designs.co.uk/website-development/php-secure-file-upload-script/
http://www.nullcandy.com/php-image-upload-security-how-not-to-do-it/
http://www.phpclasses.org/blog/post/67-PHP-security-exploit-with-GIF-images.html