Capture HTTP/HTTPS Request from Ruby (net::http, or http gem) scripts
This blog demonstrates how to capture HTTP/HTTPS traffic from Ruby scripts (net/http and http) library.
1. Make a HTTPS Request with Ruby
If you're a Ruby developer, you might spend a lot of time making an HTTP(s) Request with the following library:
- Net::HTTP: a built-in library from Ruby. It provides a rich API to help you to implement the client that uses HTTP request-response protocol.
- http: It's a 3rd-party gem. It's designed as an easy-to-use client library than the Net::HTTP.
The following sample code demonstrates how we make a request in both libraries.
require "http"
require 'net/http'
# Use net/http
uri = URI('https://httpbin.org/post?lib=net_http')
res = Net::HTTP.post_form(uri, 'isSSL' => 'true')
puts(res)
# Use http gem
res = HTTP.get("https://httpbin.org/get?lib=http").to_s
puts(res)
To run it, just save a sample as a client.rb
-> Open the Terminal app -> Execute it: ruby client.rb
.
2. How to capture HTTP/HTTPS?
Current problem
By default, both libraries (net/http and http gem) do not respect the system HTTP Proxy or HTTP_PROXY, HTTPS_PROXY variable environment. You might not see its traffic if you're using Fiddler or Charles Proxy app.
To fix it, we have to manually add the Proxy Config. Here is the config sample for the net/http
:
proxy_addr = 'localhost'
proxy_port = 9090
Net::HTTP.new('example.com', nil, proxy_addr, proxy_port).start { |http|
# always proxy via localhost:9090
}
Read more at net::http Proxy.
It's error-prone and time-consuming because we have to manually alter the source code.
Solution ✅
Luckily, Proxyman has shipped a feature to help you capture HTTP/HTTPS Requests/Responses with zero-config.
-
Download the latest macOS app at https://proxyman.io/
-
Install the Proxyman certificate on your Mac. You can simply install it by opening the Certificate menu -> Install Certificate on this Mac… -> Click on the "Install & Trust" button.
-
Open the Setup Menu -> Automatic -> Click on the "New Terminal app…"
-
New Terminal app displays. It's a pre-configured Apple Terminal app, which automatically captures and decrypts HTTPS traffic from Ruby.
-
Let's run:
ruby client.rb
again on new Terminal app, and now all HTTP/HTTPS traffic is captured and decrypted by Proxyman 🎉
- Done.
- If you'd like to understand how it works behind the scenes, check out this Automatic Setup Documentation
- If you prefer using your own Terminal (iTerm2, or Hyper), you might check out the Manual Setup.
Supported Ruby libraries
Proxyman supports: http
, net/http
, net/https
, faraday
, httparty
, and fastlane
out of the box.
Conclusion
It's error-prone and time-consuming to capture HTTP/HTTPS traffic from Ruby scripts (net/http or http gem). However, it can be done with zero-config and 1-click by using the Automatic Setup from Proxyman.
Proxyman is a high-performance macOS app, which enables developers to capture and inspect HTTP(s) traffic from apps and domains on iOS devices, iOS Simulators, and Android devices.
Get it at https://proxyman.io/