ccppi/job-scrapper
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
Only for learning purposes! The software comes with no warranty. A software that scrap jobs from different platforms puts them into a db and views them on a gui.
80 Commits 2 Branches 0 Tags 705 MiB
Python 98.6%
Makefile 1.4%
5885bf5f48
Go to file
ccppi 5885bf5f48 Update Readme.md
2024-08-23 11:10:25 +00:00
db Initial commit 2024-06-13 11:14:04 +02:00
dist/wine-nuitka update deploy binary 2024-08-13 12:21:40 +02:00
lib add force retry if first try fails, in cookie stealing 2024-08-23 13:01:20 +02:00
.gitignore sd 2024-08-08 15:03:47 +02:00
example.txt Initial commit 2024-06-13 11:14:04 +02:00
gui.spec makefile add windows build 2024-08-09 12:21:46 +02:00
makefile add zipping target for deploying 2024-08-13 12:09:09 +02:00
querry.note Initial commit 2024-06-13 11:14:04 +02:00
Readme.md Update Readme.md 2024-08-23 11:10:25 +00:00
requirements.txt makefile add windows build 2024-08-09 12:21:46 +02:00

Readme.md

Warning!!

If you run the sync to often it is very likely that your IP gets banned as bot. It is wise to use a vpn or a proxy, or just sync once or twice a day.

Dependencies

Install the requirements.txt

We depend on firefox, the reason for this is to workaround cloundflares bot protection. This works by stealing cookies from the browser.

First time usage

  1. cd lib/

Set up the database

  1. python main.py --createdb
  2. python main.py --importregiondb

Config

edit lib/conf

Main scripts

  • cd lib/

For the cmd line use:

  • python main.py --help

For the gui run:

  • python gui.py

Windows

There is a windows build in (https://5ccppi.org:3000/ccppi/job-scrapper/src/branch/threading/dist/wine-nuitka)