Tiki Crawl

A crawler tool for checking links or gathering content from websites.

https://gitlab.com/tikiwiki/tiki-crawl

Notes

• It relies on the Crawler library from Spatie (Over 5 million downloads).
• It starts off as a tool just for developers, and will eventually be integrated in Tiki, so accessible to power users.

This was used to gather data for an upcoming AI Chatbot we are working on (more news on this later, along with some code).