python 2.7 - Variable time between cron jobs (or similar implementation) -


here's idea -- there's website want scrape. updates every 10 minutes, get's out of sync. it's important information scrape before updates. each time check site, can scrape 'time remaining' until next update.

is there way make cron job -- after each iteration -- can set time wait before running the time (t+1) iteration based on variable time (t) iteration?

i'm not particularly familiar cron jobs -- current super rough implementation uses -sleep-. not ideal.

i had same problem , implementet basic checks in program scraping website. let crontab call program , program performs checks. if data in, program exits doing nothing. wastes little processing resources proved reliable solution.

entry in crontab :

    */10 * * * * cd /var/www/vhosts/website/scripts && /usr/bin/python scraper.py

you can make interval shorter make fit window of updates on site.

    */5 * * * * cd /var/www/vhosts/website/scripts && /usr/bin/python scraper.py

*/5 executes script every 5 minutes, not have "sleep" program.

i advise against manipulating crontab program/script.


Comments

Post a Comment

Popular posts from this blog

javascript - AngularJS custom datepicker directive -

i make custom datepicker directive custom template. but have no idea how start building it... how include date data directive? i appreciate guide or give me advice work on more precisely. it might you! follow below steps html code sample: <label>birth date</label> <input type="text" ng-model="birthdate" date-options="dateoptions" custom-datepicker/> <hr/> <pre>birthdate = {{birthdate}}</pre> <script type="text/ng-template" id="custom-datepicker.html"> <div class="enhanced-datepicker"> <div class="proxied-field-wrap"> <input type="text" ui-date-format="yy-mm-dd" ng-model="ngmodel" ui-date="dateoptions"/> </div> <label> <button class="btn" type="button"><i class="icon-calendar"></i>
Read more

javascript - jQuery date picker - Disable dates after the selection from the first date picker -

<!doctype html> <html lang="en"> <head> <meta charset="utf-8"> <title>jquery ui datepicker - default functionality</title> <link rel="stylesheet" href="//code.jquery.com/ui/1.11.4/themes/smoothness/jquery-ui.css"> <script src="//code.jquery.com/jquery-1.10.2.js"></script> <script src="//code.jquery.com/ui/1.11.4/jquery-ui.js"></script> <link rel="stylesheet" href="/resources/demos/style.css"> <script> $(function () { $("#datepicker1").datepicker(); }); </script> <script> $(function () { $("#datepicker2").datepicker(); }); </script> </head> <body> <p>date 1: <input type="text" id="datepicker1"></p> <p>date 2: <input type="t
Read more