Wireshark, doo doo doo doo doo doo.
Wireshark is an open-source packet analysis suite. I use it for network troubleshooting, and debugging software that uses a network to communicate.
You can even use it to monitor traffic over a bluetooth interface, or USB.
I call it a suite for the sole reason that it comes with a bunch of utilities.
In the summer of 2001, I had the pleasure of taking an SGI Indy home for the summer vacation. The problem I was to solve before school started in September was to get the password to this machine so they could use it again. They no longer had the root password--or any passwords to the machine.
The machine rain IRIX, which was really popular at the time for servers, I was convinced that I could gain access to the machine if I could find it via the IP address. Nobody patched this machine in the last few years. If you understood this reference, you probably did some things with IRIX in the late 1990s and early 2000's.
When our family got our first computer, I was 13, and we had internet the same day through my aunt. A few months later I found out that this came with shell access to the ISP, and that's when I started to learn about UNIX, because the local ISP ran IRIX.
There were a few problems:
- The monitor used a very obscure connection, and it was gigantic. I couldn't bring that home with me. It also didn't work all the time, there was something wrong with the connection.
- The keyboard was also damaged, and nothing I could connect to it would respond.
- It was nice enough to have a standard ethernet port.
What I tried:
-
My first attempt was plugging it in to the router at home which should assign an address to it via DHCP. I expected this not to work, because the original purpose of this machine was a gateway to the internet.
-
I tried plugging it into a switch to see if it would assign IP addresses to machines, it did not.
The next thing I tried, was ethereal. I had read about this on slashdot, and figured that a network
If we rewind back to the 2000s, this was wiresharks original name. I had an RTL8139 chipset in my ethernet cards which worked on almost every modern distribution of linux, including the slackware 4 I was running. Having two ethernet cards allowed me to not be blind when I connected the SGI Indy directly to the second card via acrossover cable.
With the cable connected, I started wireshark from my XFCE terminal, and pointed it at the /dev/eth1 device. My AMD Athlon 350mhz computer with a 100mhz frontside bus was quick for it's time, but this took up a lot of CPU.
A bunch of traffic started coming through. This was the first time that I had ever used this program and I was learning what everything meant. The basics of TCP were understood, I've see the words ARP before, and DNS was a thing. What is this BOOTP thing? I'll write that down for later.
A few seconds in, I stopped the trace, I had what I needed. There was a request that said:
Who has 192.168.0.3 tell 192.168.0.1
I set my IP Address to 192.168.0.3, and then sent a ping request to 192.168.0.1
It responded.
I opened up a telnet connect to 192.168.0.1, and I received the familiar IRIX telnet logon session.
At this point I took the version number it was displaying, and modified a really popular telnet exploit to work against this system, and I had root access. I used the man pages to figure out how to add users to the system, added a backup acccount, and made a setuid program so that I could get root access again if needed.
The rest of the summer I spent playing with IRIX and trying to geting the remote X-Window session working. It was a lot of fun, and I couldn't have done it without Ethereal (wireshark).
Actually, I could have, my backup plan was to connect the SCSI hard drive to my computer and use a HEX EDITOR to change the root password to a blank password, allowing me to login without a password. I used this method on a SOLARIS server I acquired a few years later from a different school board. Efficient.
This is the command line version of wireshark, and allows you to capture things "Headless".
In Windows 2012 R2+, you can capture traffic without addition utilities installed by using the netsh trace command.
@echo "Starting Trace:"
netsh trace start persistent=no capture=yes tracefile=c:\temp\nettrace.etl"
@echo "Waiting 30 seconds"
timeout 30
@echo "Stopping Trace"
netsh trace stopTo use this file in wireshark, you'll have to convert it. Thanks Benjamin Perkins: https://blogs.msdn.microsoft.com/benjaminperkins/2018/03/09/analyze-netsh-traces-with-wireshark-or-network-monitor/
The Library for Packet Capture, this is shared by a few open source projects, most people will be familiar with tcpdump or ngrep.
Using libpcap it's able to hook into network interfaces and capture traffic. Depending on your operating system, and what kind of network card(s) you have, you might be able to do some pretty advanced stuff.
You'll almost always be able to do the basics like see local traffic.
frame contains ca.yahoo.comOn Windows, using either Firefox or Chrome, you'll be able to set the following environmental variable:
SET SSLKEYLOGFILE=%USERPROFILE%\sslkeysENV.pmsIf you're curious who is broadcasting on a network, you can filter by
arpThis will display any machines that are broadcasting with the ARP protocol.
This filter will look for all SSL traffic, and find anything that is HTTP2 or HTTP.
(ssl) && ((http2) || (http))When you find a stream you are looking for, you will see 1-4 columns at the bottom of the Packet Bytes window:
- Frame
- The original data frame.
- Decrypted TLS
- The Decrypted TLS stream
- Reassembled body
- The reassembled body (all packets in this request)
- Uncompressed entity body
- If the content was compressed (it most likely was), this step will decompress it making it human readable.
Because of the hoops that have to be jumped through, I much prefer Fiddler for analyizing HTTP/HTTPS traffic.
The Meta Analysis and Tracing Engine (MATE) is my goto extension to configure on a new Wireshark install.
Examples taken from: https://wiki.wireshark.org/Mate/Examples
Cloudshark Link: http://cloudshark.org/captures/9279c75f8161
mate.tcp_ses.Time > 1This is great for narrowing down any type of sessions which don't last long. If you are looking for timeouts or other types of issues, this will be valuable.
mate.tcp_ses.NumOfPdus < 5A TCP session has a triple handshake, and usually at least a few exchnages after that. So if we search for sessions with less than 5 Packets, chances are it's an unreachable connection or something that's blocked off.
This is the (Stream Index - 1)
mate.tcp_ses == 1From the MATE documentation:
https://osqa-ask.wireshark.org/questions/10640/how-to-find-syn-not-followed-by-a-synack/10758
(mate.tcp_ses.NumOfPdus < 4) && (tcp.flags eq 0x02 && tcp.analysis.retransmission)This filter adds to the previous ones, we look for any retries, and specifically packets that are less than 4. If we make this greater than 4, we can actually find packet loss in a particular connection.
To make the most use out of the filters, chaining them becomes nessessary.
Yes
(tcp) || (dns)
No
tcp || dns
Yes
!(tcp)
No
(!tcp)
Fiddler is my goto software for HTTP Request debugging on Windows. It's always open on my computer for the off chance that I view a site that does something really iffy, and want to know more.
On the Windows Eco-system it works flawlessly and allows you to intercept traffic on the top 3 browsers.
There are a lot of programs that do things that Fiddler does, but in my humble opinion, Fiddler is king for people of all skill levels.
Burp is awesome, but I don't really want to have this installed on random servers, and gosh the IT people get in a kerfuffle when they see this. https://portswigger.net/burp
The name alone is scary, it's pretty much a tuned down version of fiddler from a GUI perspective, but incredibly powerful. https://mitmproxy.org/
Out of the box, Once fiddler starts up it should start intercepting traffic. You might have to restart your browers or web applications to see this information.
In order to receive information about web requests sent to or from a .NET Application, you can try modifying the machine.conf, the web.config, or the application.config
I'm actually pretty java dumb, but this command usually works:
java -xproxy=By default, HTTPS decryption is not enabled, for a very good reason. To enable it you must accept a root certificate that is generated by Fiddler. this root certificate allows the traffic that is intercepted from between your browser and the remote servers to be signed by something that the computer recognizes.
In Fiddler
- Open the Tools Menu, and select "Options"
- Click on the "HTTPS" tab
- Select "Capture HTTPS CONNECTs"
- Select "Decrypt HTTPS traffic"
- Click yes to every message. There may be a lot of them.
- Click on the "HTTPS" tab
This process is much less painful than Wireshark. Since Fiddler is centered around the HTTP protocol, you get a lot more information as well.
Visiting the Wookieepedia entry for R2-D2 (https://starwars.fandom.com/wiki/R2-D2) we can click on the request
Request Count: 1
Bytes Sent: 1,006 (headers:1,006; body:0)
Bytes Received: 114,097 (headers:1,039; body:113,058)
ACTUAL PERFORMANCE
--------------
ClientConnected: 21:55:13.453
ClientBeginRequest: 21:55:13.914
GotRequestHeaders: 21:55:13.914
ClientDoneRequest: 21:55:13.916
Determine Gateway: 0ms
DNS Lookup: 0ms
TCP/IP Connect: 0ms
HTTPS Handshake: 0ms
ServerConnected: 21:55:13.507FiddlerBeginRequest: 21:55:13.917
ServerGotRequest: 21:55:13.917These two metrics tell us when the Client Sent the Request, and the Server Received the Request. They are the same value, which is good performance wise.
ServerBeginResponse: 21:55:13.962This is when the server starts transferring data. If there is a large delay here, it's usually a few things:
- The Web Server
- A Backend Server (Database, File Server, etc)
- A network problem (Open up Wireshark, check for packet issues.)
GotResponseHeaders: 21:55:13.962
ServerDoneResponse: 21:55:14.145
ClientBeginResponse: 21:55:13.962
ClientDoneResponse: 21:55:14.145The time between ServerGotRequest and ServerBegingResponse is how long the request took to process
The time between ServerBeginResponse and ClientDoneResponse can be considerered the transfer time
Overall Elapsed: 0:00:00.230That's 0.23 seconds for this individual request
RESPONSE BYTES (by Content-Type)
--------------
text/html: 113,058
~headers~: 1,039The total size of the body (text/html) and the total size of the headers.
You'll notice that I have these two columns added by default in my view, as a performance person, I'm really interested in these values.
- TTFB is Time to First Byte
- The time it takes for the server to start sending data. I like this number to be as low as possible.
- TTLB is Time to Last Byte
- The time it takes for the server to send it's last byte of data. You subtract TTFB from TTLB, to get the Total Transfer Time.
You can unreliably extrapolate a transfer speed from these two values. This helps remove any network speed problems from determinig why an application is slow.
If TTLB is 5000 and TTFB is 1000 and the total bytes sent is 15,360 (15 megabytes) We can assume that it took
- 1 second to process the data.
- 4 seconds to transfer the data.
We can safely assume that the network connection is just slow. Run the request locally on the web server if possible, this removes any intermediary things like proxies or firewalls.
If TTLB is 5000 and TTFB is 4990 and the total bytes sent is 15,360 (15 megabytes) We can assume that it took
- 4.99 seconds to process the data.
- 0.01 seconds to transfer the data.
The network is probably not to blame, do an end to end inspection just in case.
Fiddler allows manual replays of Web Requests, which I feel is one of it's lesser used features.
One of my favorite parts of Fiddler, is the tamper and replay. I'll cover a very simple use case for this with twitter.
My favorite scenario was when at a previous job working for one of the provinces, we had a contractor create a client server application. They didn't put any user sanity checks into the REST apis. This meant when the application sent a permissions request, and the server sent back "User", if I changed that to "Admin", I could see all of the Administrative functions.
Although this seems like something incredibly simple, how do you test for it without writing code? Using any of the available http tamper proxies, or my favorite, fiddler.
-
Step 1: Capture and Edit the Users Recommendation request in Twitter (https://twitter.com/i/users/recommendations)
-
Step 2: Visit the likes page (https://twitter.com/i/likes)
Expected Result: I'll see 98 as the notification count.