Commit graph

1 commit

Author SHA1 Message Date
85717a7fac
feat: use flags to configure the crawler
- Use flags to configure the worker pool and the maximum number of
  pages.
- Add README.md
2024-08-27 17:11:47 +01:00