The X font server (xfs) provides a standard mechanism for an X server to communicate with a font renderer, frequently one running on a remote machine. It usually runs on TCP port 7100.
X Font Server Wikipedia
The use of server-side fonts is currently considered deprecated in favour of client-side fonts. Such fonts are rendered by the client, not by the server, with the support of the Xft2 or Cairo libraries and the XRender extension.
For the few cases in which server-side fonts are still needed, the new servers have their own integrated font renderer, so that no external one is needed. Server-side fonts can now be configured in the X server configuration files. For example, /etc/X11/xorg.conf will set the server-side fonts for Xorg.
No specification on client-side fonts is given in the core protocol.
As of October 2006, the manpage for xfs on Debian states that:
User experience show the same performance on both X server with direct font serving and X server with font server path.
So the choice between local filesystem font access and xfs-based font access is purely a local deployment choice. It does not make much sense in a single computer scenario.