DeepTrawl

Procedure

With the windows version of DeepTrawl the following sites are tested:

1. A Different End Of The Internet
2. httpstat.us
3. Rusty Links
4. Dead Link City

Findings

The average speed of this broken link checker is 8 links per second.

The maximum number of links being tested is unlimited.

Slow websites are tolerated when they respond within 20 seconds.

Unknown status codes 888 are considered as 200 Ok. LinkedIn could not be contacted.

Testing with httpstat.us gives some problems (it toke a long time, and not all expected broken links were found), however testing with Dead Link City learned that all type of errors in the 400 and 500 range are detected. Hence the related requirement is marked as passed.

Results

1. Testing with A Different End Of The Internet give these results:
   1. PASSED: Examines links to other resources (the src attribute) - Should
   2. PASSED: Examines structural links (the link tag) - Should
   3. PASSED: Respects robots.txt of the-linked-to-website - Could
   4. FAILED: Presents unchecked links due to the robots.txt directive at the-linked-to-website - Must
2. Testing with httpstat.us give these results:
   1. PASSED: Detects link error 404 - Must
   2. PASSED: Detects all link errors in the 400 and 500 range - Should
   3. FAILED: Detects redirected links - Should
3. Testing with Dead Link City give these results:
   1. PASSED: Examines links in stylesheets - Could
4. Testing with Rusty Links give these results:
   1. PASSED: Examines internal links - Must
   2. PASSED: Examines relative links in combination with the base tag - Should
   3. FAILED: Examines inline (anchor) links - Could
   4. PASSED: Examines external links - Must
   5. PASSED: Examines standard hyperlinks (the a tag) - Must
   6. PASSED: Ignores structural links with the dns-prefetch attribute - Could
   7. FAILED: Examines links with other schemes (at least FTP://) - Could
   8. PASSED: Examines Unicode links - Should
   9. FAILED: Detects blacklisted links (at least blacklisted by Google Safe Browsing) - Could
   10. FAILED: Respects robots.txt - Should
   11. PASSED: Allows you to test websites with up to 500 links - Must
   12. PASSED: Allows you to test websites with up to 2000 links - Should
   13. PASSED: Allows you to test websites with up to 5000 links - Should
   14. PASSED: Allows you to test websites without a restriction on the number of links - Should

The tests done by hand give these results:

1. PASSED: Allows you to take a subdirectory as base for the test - Should
2. PASSED: Allows you to start the test with a single click (after entering the to be tested URL) - Could
3. Not applicable: Allows you to use your mobile device - Could (only for online link checkers)
4. PASSED: Allows you to use a graphical user interface - Could (only for offline link checkers)
5. PASSED: Allows you to test an unlimited number of websites - Should
6. PASSED: Allows you to retest a website unlimited - Must
7. PASSED: Allows you to save the results (internal or by an CSV export) - Could
8. FAILED: Presents the results in one overview - Should
9. FAILED: Presents 200 Ok and broken links - Could
10. PASSED: Presents the webpage where an issue is found - Should
11. PASSED: Presents in detail (e.g. linenumber, source viewer) where an issue is found - Could
12. PASSED: Presents in a clear way the results when there are no issues found - Could
13. PASSED: Test at least 5 links per second - Could
14. FAILED: Test at least 10 links per second - Could
15. FAILED: Test at least 15 links per second - Could
16. FAILED: Test at least 20 links per second - Could
17. FAILED: Test at least 25 links per second - Could

Score

Based on the results above 64 points (out of 87) were earned. That gives a score of 74%. However unchecked links due to the robots.txt directive at the-linked-to-website are not presented. This requirement is a Must, hence this broken link checker is not advised.

The Dead Link City score is 29/75.