[Zonemaster-devel] Submitting bulk checks

Michal Toma michal.toma at afnic.fr
Wed Oct 5 07:23:04 UTC 2016


Hi Anand,

Well, my first advise would be to use a "real" database server like PostgreSQL that doesn't die because you are sending queries at it...
I admit I might be "slightly" sarcastic here, lol. I abandoned MySQL more than 10 years ago and never looked back, now I'm a PostgreSQL DBA here at Afnic administering the Afnic Labs BigData related PostgreSQL server.

So jokes apart.

First of all I'd like to konow if you are able to reproduce this issue. The perl database handles management in a perl multithreaded environment is a little bit complicated. The message might also mean that the client database handle was disconnected because of some timeout somewhere. 
If it is a problem in the code we should fix it.

The zonemaster-backend proceeds like this:
1. Schedules domains for testing with either the start_domain_test or add_batch_job APIs.
2. Inserts the test specification in JSON format to the params column of the test_results table and sets progress to 0.
3. The background runner daemon picks as many domains as the limit set by the number_of_processes_for_frontend_testing (or batch) and sets the progress to 1
4. This continues as long as there there are test_results entries with progress=0 until all tests are finished

So to reschedule the unfinished tests in case of a crash:
1. Restart the zm_wb_deamon scrip
2. Set all the tests you would like to reschedule to progress=0

In your case this should probably be:
UPDATE test_results SET progress=0 WHERE batch_id=your_batch_id_here AND progress<>100

Michal

----- Original Message -----
From: "Anand Buddhdev" <anandb at ripe.net>
To: "Michal Toma" <michal.toma at afnic.fr>
Cc: anandb at ripe.net, zonemaster-devel at lists.iis.se
Sent: Tuesday, October 4, 2016 7:19:30 PM
Subject: Re: [Zonemaster-devel] Submitting bulk checks

On 03/10/16 16:28, Michal Toma wrote:

Hi Michal,

> Attached you will find the batch_test_FR_zone.pl script that we use
> to run the zonemaster backend in batch mode.

I read the documentation of the API call "add_batch_job", and created a
job for this purpose. My JSON looks like this:

{
"method":"add_batch_job",
"params":{
	"username":"bulk",
	"api_key":"bulk",
	"test_params":{
		"ipv4":true,
		"ipv6":true
	},
	"domains":[
"200.193.193.in-addr.arpa",
"17.45.212.in-addr.arpa",
...
...
"176.188.89.in-addr.arpa",
"e.5.4.c.2.1.0.2.8.b.7.0.1.0.a.2.ip6.arpa"
]
}
}

The total number of domains to test is 721150. I submitted it like this:

curl http://localhost:5000 -d @zones.json

This ran for several seconds, and I saw results starting to appear in
the "test_results" table of the database, but then it died:

{"jsonrpc":"2.0","error":{"message":"DBD::mysql::db do failed: MySQL
server has gone away at
/opt/zonemaster/lib/perl5/Zonemaster/WebBackend/DB/MySQL.pm line
151.\n","code":-32603},"id":null}

It looks like the mysql server was killed, because it was overloaded. So
my batch job was not submitted properly, and I will have to retry. My
question is: when submitting the batch job, where does zonemaster keep a
journal of which domains are to be tested? How can it resume the testing
after the database has been restarted? The old DNScheck had a "queue"
table where all domains to be tested were queued, but zonemasters
doesn't appear to have anything like this.

Also, should I be doing anything differently when submitting a large
number of domains for testing in bulk?

Regards,
Anand


More information about the Zonemaster-devel mailing list