Scheduled Task Occasionally Launches Multiple Instances?

[rackem]

I have a Scheduled Task that cycles through and processes a large table of records in batches. The results were inconsistent and during troubleshooting I found that some records were processed more than once and nearly at the same time. This only occurred if the Task ran automatically versus manually. I think the Fabrik Scheduled Task occasionally launches multiple instances when it is triggered by a site visit.

I was able to create a workaround in my code to make sure only one instance could run at a time by first assigning a unique id to a database field, reading it back from the database, and verifying it matched, before running my desired code.

Has anyone else run across this? I assume this is not intended Fabrik behavior but also probably does not affect many people.

 

[cheesegrits]

We used to have code to prevent this, which got removed a while back, which worked by setting published=2 just before running the cron, and setting it back to 1 afterwards, so the query that checks for runnable tasks would ignore any currently still running.

https://github.com/Fabrik/fabrik/commit/55be2e958b87d7f9e63b61b40c98dd54322aa0a8

... and frankly I can't remember why we removed it. I think it was to do with crons never getting re-published if they crashed for any reason, as the code that reset published=1 would never run.

And although I made the change, I think it was after @rob convinced me to remove it.

I'll have a think, see if I can come up with a better way of doing this.

-- hugh

 

[cheesegrits]

OK, I think I've come up with a solution, using PHP's register_shutdown_function(), which will catch any fatal errors. Then in the shutdown function, I can both log the error, and re-publish the plugin.

I'll play around with it some more, and once I'm happy with it, add the code back in that makes currently running tasks un-re-runnable.

Something else I'd like to do is show the last X log results from cron jobs, on the cron admin page ...

-- hugh

 

[rackem]

Sounds great Hugh! I also really like your idea of a way to view log results versus having to go to the database.

 

[rackem]

I'll play around with it some more, and once I'm happy with it, add the code back in that makes currently running tasks un-re-runnable.

Just curious if you've committed this code Hugh? It looks like you've put in the error handling.

 

[rackem]

Another question: is it possible for me to write data from my scheduled task to the Fabrik log file?

 

[cheesegrits]

I've added the error handling, haven't done the run gate yet. It's on my list for this week.

Yes, you can log. I thought we passed in the open $log object the main cron plugin has, but we don't. So you'll have to set it up yourself:

$myLog = FabTable::getInstance('Log', 'FabrikTable');
$myLog->message_type = 'plg.cron.something.meaningful'; /// just some meaningful dotted log type
$myLog->message = "Your message here";
$myLog->store();

I think if you need to log multiple lines, you may need to do ...

$myLog->id = null;

... between stores, to avoid overwriting the same log line, although I can't recall if that's strictly necessary.

-- hugh

 

[rackem]

I'll play around with it some more, and once I'm happy with it, add the code back in that makes currently running tasks un-re-runnable.

I still seem to have multiple instances occasionally slipping through my code somehow. It would be great to have your run gate implemented.

 

[rackem]

I've taken out a new Standard subscription to put towards your work here and my other current threads Hugh. For some reason my last one was still active past it's due date. It may have been extended when I had some outstanding issues a while back.

 

[cheesegrits]

Hmm. I'll check in to that. It should have expired. I need to see if there's a bug somewhere in our expiration task.


Sent from my HTC One using Tapatalk

 

[rackem]

I noticed you've added the run gating, thank you! I've just updated from Github and am testing. So far so good but I'll re-open this if needed.

 

[cheesegrits]

That'd be good. The document you posted really doesn't make much sense to me.

-- hugh