Large quantity of tests failed

[romanovzky]

Hi there again!

First, thank you for this piece of software, it is amazing.

Second, I've been having the "Large quantity of tests failed" regularly, as in every time I use it for the past couple of weeks.

v0.4.0 (Latest)

[INFO] [nordnm.nordnm]: Downloading latest NordVPN OpenVPN configuration files to '/home/mcromao/.nordnm/configs/'.
[INFO] [nordnm.nordnm]: Checking for new connections to import...
[INFO] [nordnm.nordnm]: Benchmarking 420 servers...
[INFO] [nordnm.nordnm]: Benchmarked 127 servers successfully (30.24%). Took 3.22 seconds.
[WARNING] [nordnm.nordnm]: A large quantity of tests failed. You might want to check the reliability of your network.
[INFO] [nordnm.nordnm]: Removing all active connections...
[INFO] [nordnm.nordnm]: Adding new connections...
[INFO] [nordnm.nordnm]: 4 new connections added.
[INFO] [nordnm.networkmanager]: DNS leak protection enabled.

Has something changed? I'm running Arch-Linux (gnome desktop, if that has any significance). I get the same results on different computers (both cable and wi-fi setups tested, etc) running the same version and Arch Linux.

Cheers!

[chadsr]

This warning is actually new in v0.4.0 which was released more-or-less a couple of weeks ago, so that is why you are only just seeing it :).

This issue is likely related to #75 and the latest reply I have made regarding ICMP flooding protection. It seems that the networks I am usually on do not have this flooding protection, so I have been missing the severity of the problem witnessed by a lot of users.

In that post I have proposed some possible solutions and hope to have some feedback before implementing a potential work-around.

[romanovzky]

Hi there!
I read #75 and if you recall I had a similar issue before (#69) about the ICMP flooding protection. I have since then deactivated the ICMP flood protection in my router, and I can't do any other workaround about it... I guess my ISP imposes some sort of flood protection outside my realm of influence...
I'm looking forward some of the proposed solutions in #75 , namely the possibility to ping only the top few servers or add a slow-mode flag.
Cheers

[chadsr]

Hi again.

An experimental 'slow-mode' will be included in the next release (5f645da). Since I am unable to test its effectiveness on a large variety of networks, some feedback will probably be needed to see if it works as intended and increases the percentage success.

I had began implementing a "benchmarking limit" feature, so it was possible to specify how many of the "top" servers to test but it occurred to me that defining what a "top" server is, is kind of problematic...
I see similar tools using the percentage load, so only the 10 lowest loaded servers, for example are tested (Which doesn't seem very useful, if better servers are usually more full).
The official NordVPN application seems to use geographical location for a similar purpose. (Probably better than the previous example, but still not really that accurate)

So yeah, I halted on the "benchmark only the top x servers" approach for now, as I am worried about the tool finding very sub-optimal servers with heuristics like those above. Some further experimenting to define what "top" servers are is definitely needed.

[romanovzky]

Cheers! I'm looking forward to see it in action and to provide feedback. I'm on Arch and the package seems to be updated fairly quickly after you release it, so I'll keep in touch soon.

[chadsr]

Version 0.5.0 is now available and contains a "slow-mode" argument for you to test out. You can now simply use it like so: sync -s along with any other sync arguments you normally use.
With this option we should be able to tell if the problem does simply boil down to some overlord/ISP blocking large scale ICMP requests. :)

I don't consider this option to be a final solution and am currently working on a rather big overhaul, which should remove this problem totally.
The overhaul will make use of better heuristics to reduce the number of servers which need pinging, or possibly just utilising a reverse engineered endpoint from NordVPN to find the 'best servers', which @dapolinario pointed out in #32 .

[bocekm]

I have used the slow mode in the v0.5.1 and interestingly, the tests do not fail, but still very few connections are added.

• with 5 pings:

[INFO] [nordnm.nordnm]: Benchmarking slow mode enabled.
[INFO] [nordnm.nordnm]: Benchmarking 4016 servers...
[INFO] 4016/4016 benchmarks finished.
[INFO] [nordnm.nordnm]: Benchmarked 3935 servers successfully (97.98%). Took 983.56 seconds.
[INFO] [nordnm.nordnm]: No active connections to remove.
[INFO] [nordnm.nordnm]: Adding new connections...
[INFO] [nordnm.nordnm]: 70 new connections added.

• with 1 ping:

[INFO] [nordnm.nordnm]: Benchmarking slow mode enabled.
[INFO] [nordnm.nordnm]: Benchmarking 4017 servers...
[INFO] 4017/4017 benchmarks finished.
[INFO] [nordnm.nordnm]: Benchmarked 3955 servers successfully (98.46%). Took 514.09 seconds.
[INFO] [nordnm.nordnm]: Removing all active connections...
[INFO] [nordnm.nordnm]: Adding new connections...
[INFO] [nordnm.nordnm]: 71 new connections added.

[romanovzky]

I can confirm this solved the high benchmark failures at the expense of running time. I'm fine with this, so thanks!

[IgnusG]

For me this fixed it as well. Went from around 3% success rate to 97% whilst running for around a minute and a half which is still really good.

Thanks for that fix!

[chadsr]

Good to see the slow-mode has generally solved the problem for now. As mentioned before, I'm working on some optimisations that will reduce the time taken benchmarking significantly. I'm combining that update with quite some other overhauling, so it may take some time to get released.

@bocekm, may I ask what your settings are? 70-71 connections could well be the number of connections the tool needs to provide a connection for all required combinations. If I can see the contents of your ~/.nordnm/settings.conf it will be easy to tell if there is a problem here, or if the connections added is correct.

Thanks!

[bocekm]

$ cat ~/.nordnm/settings.conf
[Countries]
# simply write country codes separated by spaces e.g. country-blacklist = gb us
country-blacklist = 

# same as above. if this is non-empty, the blacklist is ignored
country-whitelist = 

[Categories]
anti-ddos = false
dedicated-ip-servers = true
double-vpn = false
onion-over-vpn = false
p2p = true
standard-vpn-servers = true

[Protocols]
tcp = true
udp = false

[DNS]
custom-dns-servers = 

[Benchmarking]
ping-attempts = 1

[chadsr]

@bocekm So from playing around with the settings you are using, I think around 70 connections is correct.

Maybe I should expand the README to make it clearer what exactly the tool is importing. The tool is only importing the 'best' server for each configuration combination (us, normal, tcp). If combinations share the same server, then only one shared connection is imported, which is why the number of connections added varies sometimes.

So, there are a total of 62 available countries at the moment. Only TCP is enabled and the categories dedicated, p2p, normal.
Standard and P2P servers are often the same, giving us around 62 connections, give or take. Not many countries support servers with dedicated IP, so we get around 4-5 dedicated IP connections (NL, GB, US DE, ...). In total, the lower estimate would then be something like 66, where the other 4 or 5 added in your case are when P2P and normal do not share the same server.

So in conclusion, the number of connections you are getting seems to be correct. If you still have some confusion about how the tool selects these, then I can try to elaborate further.

[peteruithoven]

The following issue might also have some interesting discussions: #37

[bocekm]

@chadsr, understood, thank you.