Diagnosis of crawl versus replay problems

[anjackson]

A common issue is that it is not clear if a problem with a site is due to gaps in the crawl, or replay-time rewriting limitations. It should be possible to use proxy playback mode to evaluate the crawl without the additional complications of replay-time rewrites. There's two strands to this:

• Making it easier for users to check things manually in proxy mode (probably best done via a browser extension?)
• Automated diagnosis of rewriting issues, by comparing proxy-mode playback with re-write mode playback.

[jkafader]

Another question in this sort of area is "can we trace a missing DOM node back to a particular missing URL?" - which would be useful if what we're trying to eventually do is actually get a crawler to go and fix the page in an automated way.