Web Scraping
Hi, i'm doing web scraping on professional basis. See my CV here
- Rust and Python languages
- Messages and storage with MongoDB, Redis and gRPC
Web scraping, also known as "web harvesting" or "web data extraction," is a method used to extract valuable information from diverse sources in a structured format, making it suitable for further utilization.
My typical projects encompass the development of web robots, data processing systems, REST services, and SPA dashboards.
- Collecting data to train domain specific llms.
- Collecting data for vairous kind of AI systems.
I employ cutting-edge technologies to ensure top-notch results for my clients.
Projects:
-
Rust implementation of Fei Sun, Dandan Song and Lejian Liao paper Content Extraction via Text Density (CETD) https://github.com/oiwn/dom-content-extraction
-
Rust implementation of Stochastic Room Impulse Response Generation. https://github.com/oiwn/stoRIR-rs