Get to windows folder options by clicking View>Folder Options on the windows explorer menu.

Click the View tab.

Now uncheck the option "Hide file extensions for known file types"

Click OK and you should now see the file extensions for all files.