ESP32 network discovery through LAN IP scanning and Ethernet ARP monitoring

Recap

After power cycling Fritz and Libero [1], Libero starts obtaining addresses starting at x.20 again.

ARP table

? (192.168.178.20) at 80:7d:3a:c2:de:44 on en1 ifscope [ethernet]
? (192.168.178.22) at 80:7d:3a:c2:de:44 on en1 ifscope [ethernet]
? (192.168.178.24) at 80:7d:3a:c2:de:44 on en1 ifscope [ethernet]
? (192.168.178.26) at 80:7d:3a:c2:de:44 on en1 ifscope [ethernet]
? (192.168.178.29) at 80:7d:3a:c2:de:44 on en1 ifscope [ethernet]
? (192.168.178.31) at 80:7d:3a:c2:de:44 on en1 ifscope [ethernet]
? (192.168.178.33) at 80:7d:3a:c2:de:44 on en1 ifscope [ethernet]

nmap

Looping manually

$ nmap -p 23 -oG - 192.168.178.0/24
# Nmap 7.70 scan initiated Thu Jul  4 15:39:08 2019 as: nmap -p 23 -oG - 192.168.178.0/24
Host: 192.168.178.1 (fritz.box)	Status: Up
Host: 192.168.178.1 (fritz.box)	Ports: 23/closed/tcp//telnet///
Host: 192.168.178.33 ()	Status: Up
Host: 192.168.178.33 ()	Ports: 23/open/tcp//telnet///
Host: 192.168.178.35 (fritz.repeater)	Status: Up
Host: 192.168.178.35 (fritz.repeater)	Ports: 23/closed/tcp//telnet///
Host: 192.168.178.101 (offgrid.fritz.box)	Status: Up
Host: 192.168.178.101 (offgrid.fritz.box)	Ports: 23/closed/tcp//telnet///
# Nmap done at Thu Jul  4 15:39:10 2019 -- 256 IP addresses (4 hosts up) scanned in 1.36 seconds

Hit on x.33 after ~1,5 scanning time:

$ nmap -p 23 -oG - 192.168.178.0/24
Host: 192.168.178.33 ()	Status: Up
Host: 192.168.178.33 ()	Ports: 23/open/tcp//telnet///
# Nmap done at Thu Jul  4 15:39:10 2019 -- 256 IP addresses (4 hosts up) scanned in 1.36 seconds

  1. … and clearing out the ARP table on Offgrid ↩︎

nmap performance options

Looking at you.

TIMING AND PERFORMANCE:
  Options which take <time> are in seconds, or append 'ms' (milliseconds),
  's' (seconds), 'm' (minutes), or 'h' (hours) to the value (e.g. 30m).
  -T<0-5>: Set timing template (higher is faster)
  --min-hostgroup/max-hostgroup <size>: Parallel host scan group sizes
  --min-parallelism/max-parallelism <numprobes>: Probe parallelization
  --min-rtt-timeout/max-rtt-timeout/initial-rtt-timeout <time>: Specifies
      probe round trip time.
  --max-retries <tries>: Caps number of port scan probe retransmissions.
  --host-timeout <time>: Give up on target after this long
  --scan-delay/--max-scan-delay <time>: Adjust delay between probes
  --min-rate <number>: Send packets no slower than <number> per second
  --max-rate <number>: Send packets no faster than <number> per second

Outcome

Host probing by ping scan

$ sudo nmap -sn --min-parallelism 128 -oG - 192.168.178.0/24
Warning: Your --min-parallelism option is pretty high!  This can hurt reliability.
# Nmap 7.70 scan initiated Thu Jul  4 15:51:00 2019 as: nmap -sn --min-parallelism 128 -oG - 192.168.178.0/24
Host: 192.168.178.1 (fritz.box)	Status: Up
Host: 192.168.178.34 (andromeda.fritz.box)	Status: Up
Host: 192.168.178.35 (fritz.repeater)	Status: Up
Host: 192.168.178.53 ()	Status: Up
Host: 192.168.178.101 (offgrid.fritz.box)	Status: Up
# Nmap done at Thu Jul  4 15:51:01 2019 -- 256 IP addresses (5 hosts up) scanned in 0.68 seconds

Port probing, with Host discovery

$ nmap -p 23 --min-parallelism 128 -oG - 192.168.178.0/24
Warning: Your --min-parallelism option is pretty high!  This can hurt reliability.
# Nmap 7.70 scan initiated Thu Jul  4 15:52:32 2019 as: nmap --min-parallelism 128 -p 23 -oG - 192.168.178.0/24
Host: 192.168.178.1 (fritz.box)	Status: Up
Host: 192.168.178.1 (fritz.box)	Ports: 23/closed/tcp//telnet///
Host: 192.168.178.35 (fritz.repeater)	Status: Up
Host: 192.168.178.35 (fritz.repeater)	Ports: 23/closed/tcp//telnet///
Host: 192.168.178.55 ()	Status: Up
Host: 192.168.178.55 ()	Ports: 23/open/tcp//telnet///
Host: 192.168.178.101 (offgrid.fritz.box)	Status: Up
Host: 192.168.178.101 (offgrid.fritz.box)	Ports: 23/closed/tcp//telnet///
# Nmap done at Thu Jul  4 15:52:33 2019 -- 256 IP addresses (4 hosts up) scanned in 1.27 seconds

Isn’t this a function of DHCP? We can recommend to assign always the same IP. This is possible in the Fritzbox settings, but of cause no general solution.

Looks like DHCP lease times are not honored at all.

That’s true. Static DHCP assignments – while tempting – is nothing we should put our feet on.

May I humbly ask @MKO, @clemens or @tonke to check whether this will work on a WSL2’ed Debian/Ubuntu and tell me about its outcome? I’m actually interested in success and speed. Thanks already!

P.S.: If this works and you feel adventurous, you might want to try speeding things up:

nmap -p 23 --min-parallelism 128 -oG - 192.168.178.0/24

I tested it several times, all with the same result. I think it does not work on WSL.

After that, i have install nmap for Windows

1 Like

Dear @MKO,

thanks for testing.

Sorry, I just recognized there might have slipped something through my guidelines. Did you also try adding sudo to the command like

sudo nmap -p 23 -oG - 192.168.178.0/24

?

However, as

seems to work at last, we know it could be an option to use.

Saying that, I’m in fact looking actually at Scapy. They are talking about that the most recent version would also work on Windows, see Download and Installation — Scapy 2.5.0 documentation.

$ pip install scapy

$ sudo scapy

                     aSPY//YASa
             apyyyyCY//////////YCa       |
            sY//////YSpcs  scpCY//Pp     | Welcome to Scapy
 ayp ayyyyyyySCP//Pp           syY//C    | Version 2.4.0
 AYAsAYYYYYYYY///Ps              cY//S   |
         pCCCCY//p          cSSps y//Y   | https://github.com/secdev/scapy
         SPPPP///a          pP///AC//Y   |
              A//A            cyP////C   | Have fun!
              p///Ac            sC///a   |
              P////YCpc           A//A   | We are in France, we say Skappee.
       scccccp///pSP///p          p//Y   | OK? Merci.
      sY/////////y  caa           S//P   |             -- Sebastien Chabal
       cayCyayP//Ya              pY/Ya   |
        sY/PsY////YCc          aC//Yp
         sc  sccaCY//PCypaapyCP//YSs
                  spCPY//////YPSps
                       ccaacs

>>> arping('192.168.178.0/24')
Begin emission:
***.*.Finished sending 256 packets.
.
Received 7 packets, got 4 answers, remaining 252 packets
  bc:05:43:e0:15:4d 192.168.178.1
  04:d6:aa:84:d5:1f 192.168.178.34
  80:7d:3a:c2:de:44 192.168.178.154
  0e:96:d7:cf:cb:ea 192.168.178.35
(<ARPing: TCP:0 UDP:0 ICMP:0 Other:4>, <Unanswered: TCP:0 UDP:0 ICMP:0 Other:252>)

The Scapy blueprint for a Simplistic ARP Monitor looks promising.

Minimal ARP monitor in Python

Install

virtualenv .venv3
source .venv3/bin/pip
pip install scapy

arpmon.py

Put this into a file called "arpmon.py".

#! /usr/bin/env python
import sys
from scapy.all import *


def arp_monitor_callback(pkt):
    if ARP in pkt and pkt[ARP].op in (1,2): #who-has or is-at
        print(pkt.sprintf("%ARP.hwsrc% %ARP.psrc%"))
        sys.stdout.flush()


sniff(prn=arp_monitor_callback, filter="arp", store=0)

Invoke

sudo python arpmon.py | grep '80\:7d\:3a'

Result

80:7d:3a:c2:de:44 0.0.0.0
80:7d:3a:c2:de:44 0.0.0.0
80:7d:3a:c2:de:44 0.0.0.0
80:7d:3a:c2:de:44 192.168.178.174
80:7d:3a:c2:de:44 192.168.178.174
80:7d:3a:c2:de:44 192.168.178.174
80:7d:3a:c2:de:44 192.168.178.174
80:7d:3a:c2:de:44 192.168.178.174

on su, or with sudo it dosn´t work.


I have testet arp -a on windows comand line as admin, too.
After deleting the cache. I wait only 1 transmission
image

for /l %i in (1,1,255) DO @ping 192.168.178.%i -n 1 | find "Bytes=" 

works sometimes too, but very … verry … slowly.
tzzzzzz…

Arp on Win Comand line with 2 Fipy
arp -a | find “80-7d-3a”

image

Sorry to hear that. arpmon.py for Scapy feels reasonably snappy for me, so you might want to try that route. I am looking at this as we can integrate it into our tooling more seamlessly than having to parse output of any other command line tool.

sorry for the misunderstanding.
I mean, my ping in a loop on all ip. With this command:
for /l %i in (1,1,255) DO @ping 192.168.178.%i -n 1 | find “Bytes=”

Arp works fast on Windows command
arp -a | find “80-7d-3a”

Arp on Python i have not checked.

All good, no misunderstanding at all. Following regular ICMP ping requests will always be a magnitude slower, especially when doing that sequentially from userspace.

Right, that’s why I would favor it over actually scanning the network. The Python thing is just that: Watching the network for ARP events. If we could get the basic version work on Windows, I would gain more confidence in going down that route.

I also like this as it will read ARP discovery packages directly from the network probably without touching the operation system’s ARP table at all which sometimes confuses me.

Maybe I’m not understanding the problem but wouldn’t mDNS not solve the issue?

Sure. We are tracking this over there.

… just still following both variants to be able to tell them apart.

:) just found the other thread AFTER posting here

i tried in WSL2, looks not so good.



Unfortunately I do not have enough time now, but I’ll see if I can fix it.
Under Windows directly, I seem to have a damaged installation of python. I will try it there again.

Shit happens. Sorry for that.

No worries, please take your time and thanks already.

Some hints for the next iteration.


So yes: Trying to do this on WSL2 will probably not work as the foundation is based on Npcap which will be installed natively on Windows.

Download and Installation — Scapy 2.5.0 documentation
Npcap: Windows Packet Capture Library & Driver