All Administrators have remote access to the server by default. Restricting access using the System applet will not help as this does not override all Administrators.
If you truly want only the Administrator account to have privileges to connect, I would edit the TS connection object in Terminal Services Configuration.
Start > Run > tscc.msc > OK. Expand 'Connections' and right-click to edit the properties on RDP-Tcp. Go to Permissions and remove the Administrators group and the Remote Desktop Users group. Add in the 'Administrator' account and grant Full Control, then Apply/OK.
I would suggest you verify you do this at the server console, or you give yourself an alternate means of logging in (VNC) and TEST it works, prior to making this change; if you make it remotely and make a mistake, you may kill your session and be unable to log back in.
-Matt