This is a collection of tips and best practices I had the misfortune to learn the hard way while debugging a recent tile cache process.
Some of the best advice I saw out there suggested that the faster you can get your layers to draw the faster the cache will be created.
A couple of ways to improve drawing speed:
For a variety of reasons, it’s often advisable to have your data in an ArcSDE geodatabase when serving this data via ArcServer. In the case of tile cache creation, however, it appears that a local (to the server) file geodatabase is the way to go. This is due to the thousands (millions?, billions?!) of requests the caching tool service is making to the data while the tiles are being created. An ArcSDE instance backed by a database of some sort (Postgres, SQL Server, etc.) isn’t optimized for this sort of data exchange.
(As a note, these databases excel at tabular and geometric functions, and versioning management, and probably many other things, just not speedy data requests; though this could be an ArcSDE thing).
So the bottom line, is put your data in a file geodatabase locally on the server to create the cache, if you can. From there you can export the cache files to wherever you need them.
When selecting the number of instances to devote to tile cache creation there is some advice out there (Allocation of server resources to caching), though I had trouble making much sense of it. I think this really becomes an issue when you cluster servers to cache (or serve) large sets of data in parallel. One issue I encountered however, is that when working with SQL Express you must limit the number of instances operating in parallel to two (http://resources.arcgis.com/en/help/main/10.2/index.html#//001w0000004m000000).
Rebuilding Cache Status
When debugging a cache failure, a certain set of tools can be helpful. These are located under Server Tools > Caching > Manage Map Server Cache Status. Occasionally, I found that when I viewed the status progress of a tile cache creation, it would display funky results. Sometimes it would say “In Progress”, but no levels were showing any progress; other times it would say in progress overall and at multiple levels but appear to stall out.
One option within this tool is to rebuild the cache status. It’s like a reset on where the tile cache situation is actually at. From here you can right click on your tile cache > Manage Cache > Manage Tiles and then select Rebuild Empty Tiles from within the tool.
To go one step further use the Report Bundle Status option from within the manage cache status tool. This tool outputs a file gdb containing features with information regarding the tile cache status for each location. I didn’t go this far, but it sounds promising: Spatial reporting of map cache status