ESP32 network discovery through LAN IP scanning and Ethernet ARP monitoring

MKO · July 5, 2019, 3:05pm

I tested it several times, all with the same result. I think it does not work on WSL.

After that, i have install nmap for Windows

Andreas · July 5, 2019, 3:34pm

Dear @MKO,

thanks for testing.

Sorry, I just recognized there might have slipped something through my guidelines. Did you also try adding sudo to the command like

sudo nmap -p 23 -oG - 192.168.178.0/24

?

However, as

seems to work at last, we know it could be an option to use.

Saying that, I’m in fact looking actually at Scapy. They are talking about that the most recent version would also work on Windows, see Download and Installation — Scapy 2.5.0 documentation.

Andreas · July 5, 2019, 3:53pm

$ pip install scapy

$ sudo scapy

                     aSPY//YASa
             apyyyyCY//////////YCa       |
            sY//////YSpcs  scpCY//Pp     | Welcome to Scapy
 ayp ayyyyyyySCP//Pp           syY//C    | Version 2.4.0
 AYAsAYYYYYYYY///Ps              cY//S   |
         pCCCCY//p          cSSps y//Y   | https://github.com/secdev/scapy
         SPPPP///a          pP///AC//Y   |
              A//A            cyP////C   | Have fun!
              p///Ac            sC///a   |
              P////YCpc           A//A   | We are in France, we say Skappee.
       scccccp///pSP///p          p//Y   | OK? Merci.
      sY/////////y  caa           S//P   |             -- Sebastien Chabal
       cayCyayP//Ya              pY/Ya   |
        sY/PsY////YCc          aC//Yp
         sc  sccaCY//PCypaapyCP//YSs
                  spCPY//////YPSps
                       ccaacs

>>> arping('192.168.178.0/24')
Begin emission:
***.*.Finished sending 256 packets.
.
Received 7 packets, got 4 answers, remaining 252 packets
  bc:05:43:e0:15:4d 192.168.178.1
  04:d6:aa:84:d5:1f 192.168.178.34
  80:7d:3a:c2:de:44 192.168.178.154
  0e:96:d7:cf:cb:ea 192.168.178.35
(<ARPing: TCP:0 UDP:0 ICMP:0 Other:4>, <Unanswered: TCP:0 UDP:0 ICMP:0 Other:252>)

Andreas · July 5, 2019, 4:00pm

The Scapy blueprint for a Simplistic ARP Monitor looks promising.

Andreas · July 5, 2019, 4:24pm

Minimal ARP monitor in Python

Install

virtualenv .venv3
source .venv3/bin/pip
pip install scapy

`arpmon.py`

Put this into a file called "arpmon.py".

#! /usr/bin/env python
import sys
from scapy.all import *


def arp_monitor_callback(pkt):
    if ARP in pkt and pkt[ARP].op in (1,2): #who-has or is-at
        print(pkt.sprintf("%ARP.hwsrc% %ARP.psrc%"))
        sys.stdout.flush()


sniff(prn=arp_monitor_callback, filter="arp", store=0)

Invoke

sudo python arpmon.py | grep '80\:7d\:3a'

Result

80:7d:3a:c2:de:44 0.0.0.0
80:7d:3a:c2:de:44 0.0.0.0
80:7d:3a:c2:de:44 0.0.0.0
80:7d:3a:c2:de:44 192.168.178.174
80:7d:3a:c2:de:44 192.168.178.174
80:7d:3a:c2:de:44 192.168.178.174
80:7d:3a:c2:de:44 192.168.178.174
80:7d:3a:c2:de:44 192.168.178.174

MKO · July 5, 2019, 4:55pm

on su, or with sudo it dosn´t work.

I have testet arp -a on windows comand line as admin, too.
After deleting the cache. I wait only 1 transmission

for /l %i in (1,1,255) DO @ping 192.168.178.%i -n 1 | find "Bytes="

works sometimes too, but very … verry … slowly.
tzzzzzz…

Arp on Win Comand line with 2 Fipy
arp -a | find “80-7d-3a”

Andreas · July 5, 2019, 5:07pm

Sorry to hear that. arpmon.py for Scapy feels reasonably snappy for me, so you might want to try that route. I am looking at this as we can integrate it into our tooling more seamlessly than having to parse output of any other command line tool.

MKO · July 5, 2019, 5:40pm

sorry for the misunderstanding.
I mean, my ping in a loop on all ip. With this command:
for /l %i in (1,1,255) DO @ping 192.168.178.%i -n 1 | find “Bytes=”

Arp works fast on Windows command
arp -a | find “80-7d-3a”

Arp on Python i have not checked.

Andreas · July 5, 2019, 5:59pm

All good, no misunderstanding at all. Following regular ICMP ping requests will always be a magnitude slower, especially when doing that sequentially from userspace.

Right, that’s why I would favor it over actually scanning the network. The Python thing is just that: Watching the network for ARP events. If we could get the basic version work on Windows, I would gain more confidence in going down that route.

I also like this as it will read ARP discovery packages directly from the network probably without touching the operation system’s ARP table at all which sometimes confuses me.

poesel · July 5, 2019, 8:03pm

Maybe I’m not understanding the problem but wouldn’t mDNS not solve the issue?

Andreas · July 5, 2019, 8:07pm

Sure. We are tracking this over there.

… just still following both variants to be able to tell them apart.

poesel · July 5, 2019, 8:10pm

:) just found the other thread AFTER posting here

MKO · July 5, 2019, 8:10pm

i tried in WSL2, looks not so good.

Unfortunately I do not have enough time now, but I’ll see if I can fix it.
Under Windows directly, I seem to have a damaged installation of python. I will try it there again.

Andreas · July 5, 2019, 8:30pm

Shit happens. Sorry for that.

No worries, please take your time and thanks already.

Andreas · July 5, 2019, 8:52pm

Some hints for the next iteration.

So yes: Trying to do this on WSL2 will probably not work as the foundation is based on Npcap which will be installed natively on Windows.

– Download and Installation — Scapy 2.5.0 documentation
– Npcap: Windows Packet Capture Library & Driver

MKO · July 5, 2019, 9:49pm

I went for Python on Windows with Anaconda. So the easy way, without PATH problems to the libraries.

Michael Yin:
Some articles online trying to teach people to install python, modify PATH env, I must say this method is very complex and it might fail on another windows version, so try to use anaconda, which would save you a lot of time.

It works!

(base) C:\Users\Büro1>python arpmon.py | find "80:7d:3a"
80:7d:3a:c3:39:1c 0.0.0.0
80:7d:3a:c3:39:1c 0.0.0.0
80:7d:3a:c3:39:1c 0.0.0.0
80:7d:3a:c3:39:1c 192.168.178.160
80:7d:3a:c3:39:1c 192.168.178.160
80:7d:3a:c3:39:1c 192.168.178.160
80:7d:3a:c3:39:1c 192.168.178.160
80:7d:3a:c3:39:1c 192.168.178.160

There is still an error message or info in scapy:

Andreas · July 5, 2019, 9:59pm

Excellent, thanks. Don’t worry about the info output of the scapy program, we will just use it as a library.

So, essentially you have to install Anaconda, Npcap and Scapy ^[1] natively on Windows to get things going – right?

by invoking like conda install scapy? ↩︎

MKO · July 5, 2019, 10:35pm

yes, just install anaconda and then scapy with :

conda install -c conda-forge scapy

I did not need to install Npcap additionally.

But i will test, on an another computer again, from the beginning, at first without Anaconda.
My windows and not phyton are on this System damage, I don´t come to the path settings. Probably it was damaged during the update to win 10.

Andreas · July 10, 2019, 10:40am

Outcome

We have been able to make some way into an implementation and described the outcome within
Terkin Agent and Wartungsmodus für den Terkin-Datenlogger.

Synopsis

Displays successful device discovery and maintenance mode enablement.

$ sudo python3 -m tools.terkin maintain

2019-07-10 19:49:21,698 [tools/terkin.py] INFO   : IP networks found: ['192.168.178.0/24']
2019-07-10 19:49:21,698 [tools/terkin.py] INFO   : Sending an ARP ping to discover already connected devices on network 192.168.178.0/24
2019-07-10 19:49:21,698 [tools/terkin.py] INFO   : Waiting for any devices having MAC address prefixes of ['30:ae:a4', '80:7d:3a'] to appear on your local network
2019-07-10 19:49:25,172 [tools/terkin.py] INFO   : Found device at {'mac': '80:7d:3a:c2:de:44', 'ip': '192.168.178.166'}
2019-07-10 19:49:25,172 [tools/terkin.py] INFO   : Connecting to device mode server at 192.168.178.166:666
2019-07-10 19:49:25,172 [tools/terkin.py] INFO   : Pulling 192.168.178.166 into maintenance mode

Andreas · July 12, 2019, 3:27pm

A post was merged into an existing topic: Wartungsmodus für den Terkin-Datenlogger