Changing the screen resolution on Server Core is not likely to be a day-to-day task for most administrators. Microsoft did not bother creating a nice tool like SCONFIG in the R2 version of the software to help us with this. To change the screen resolution in Server Core, you have to manually edit the registry.
On your Server Core, type regedit and press Enter. You will be presented with the graphical Registry Editor.
Expand each GUID until you find the one that has a subkey labeled VolitileSettings.
Click the 0000 Key above the VolitileSettings subkey.
Double click the DefaultSettings.XResolution.
Enter the number of horizontal pixels you want and click OK.
Repeat the process for the DefaultSettings.YResolution
I choose 640x480 for the resolution that I want.
Close the Registry Editor. Log off and then log back on to get your new resolution.