web-crawler

Author	SHA1	Message	Date
Dan Anglin	e2cb65dc03	refactor: replace internal with linkType in record struct	2024-08-28 14:51:34 +01:00
Dan Anglin	caa6bbfe7e	feat: generate CSV reports and save to file The crawler can now generate CSV reports and save both text and CSV reports to a file.	2024-08-28 12:00:25 +01:00
Dan Anglin	5498ac7b4e	feat: add external links to the report All checks were successful Tests / test (pull_request) Successful in 15s Details	2024-08-28 07:39:24 +01:00
Dan Anglin	85717a7fac	feat: use flags to configure the crawler - Use flags to configure the worker pool and the maximum number of pages. - Add README.md	2024-08-27 17:11:47 +01:00
Dan Anglin	4519de764e	feat: add the web crawler All checks were successful Tests / test (pull_request) Successful in 13s Details Add the source code for the web crawler. The web crawler is a simple Go CLI application that traverses through a website and generates a report of all the internal links found in the site.	2024-08-27 15:42:26 +01:00