2024-08-23 11:10:25 +00:00
|
|
|
# Warning!!
|
|
|
|
|
If you run the sync to often it is very likely that your IP gets banned as bot. It is wise to use a vpn or a proxy, or just sync once or twice a day.
|
|
|
|
|
|
2024-06-13 09:39:00 +00:00
|
|
|
# Dependencies
|
2024-08-23 06:30:35 +00:00
|
|
|
Install the requirements.txt
|
2024-08-23 06:30:56 +00:00
|
|
|
|
2024-08-23 06:30:18 +00:00
|
|
|
We depend on firefox, the reason for this is to workaround cloundflares bot protection. This works by stealing cookies from the browser.
|
2024-06-13 09:37:35 +00:00
|
|
|
|
2024-06-13 09:29:08 +00:00
|
|
|
# First time usage
|
2024-06-26 11:26:42 +00:00
|
|
|
1. cd lib/
|
2024-06-14 09:33:05 +00:00
|
|
|
|
2024-06-26 11:26:42 +00:00
|
|
|
### Set up the database
|
|
|
|
|
|
|
|
|
|
2. python main.py --createdb
|
|
|
|
|
3. python main.py --importregiondb
|
2024-06-13 09:30:07 +00:00
|
|
|
|
2024-06-13 09:29:08 +00:00
|
|
|
# Config
|
2024-06-26 11:24:55 +00:00
|
|
|
edit lib/conf
|
2024-06-13 09:25:28 +00:00
|
|
|
|
|
|
|
|
|
2024-06-13 09:29:08 +00:00
|
|
|
# Main scripts
|
2024-06-26 11:24:33 +00:00
|
|
|
- cd lib/
|
|
|
|
|
|
|
|
|
|
For the cmd line use:
|
2024-06-26 11:22:49 +00:00
|
|
|
- python main.py --help
|
2024-06-26 11:24:33 +00:00
|
|
|
|
2024-06-26 11:22:49 +00:00
|
|
|
For the gui run:
|
|
|
|
|
- python gui.py
|
2024-06-13 09:25:28 +00:00
|
|
|
|
2024-08-09 12:57:26 +00:00
|
|
|
# Windows
|
|
|
|
|
There is a windows build in (https://5ccppi.org:3000/ccppi/job-scrapper/src/branch/threading/dist/wine-nuitka)
|
