job-scrapper/Readme.md

Warning!!

If you run the sync to often it is very likely that your IP gets banned as bot. It is wise to use a vpn or a proxy, or just sync once or twice a day.

Dependencies

Install the requirements.txt

We depend on firefox, the reason for this is to workaround cloundflares bot protection. This works by stealing cookies from the browser.

First time usage

1. cd lib/

Set up the database

2. python main.py --createdb
3. python main.py --importregiondb

Config

edit lib/conf

Main scripts

• cd lib/

For the cmd line use:

• python main.py --help

For the gui run:

• python gui.py

Windows

There is a windows build in (https://5ccppi.org:3000/ccppi/job-scrapper/src/branch/threading/dist/wine-nuitka)