I was introduced to software development during a coding Bootcamp in 2018. Over there, I was amazed by the simplicity of the Ruby on Rails framework, especially the MVC pattern. It was very simple: the model was in charge of communicating with the database and retrieving all the necessary data. The controller was responsible for responding to user actions, asking the model for the data the user was requesting, and sending it to the view, which was responsible for rendering what the user would see in the browser simple and plain HTML.

Suddenly how the web worked made more sense to me. After a couple of projects, I started to want more interactivity in my websites (after all, I was a user of other products, and the bar for quality was high). That was how I met JavaScript and AJAX. I was fortunate enough to learn JS in the ES6 era, so I was able to achieve most of the things I wanted with sprinkles of vanilla JavaScript. With time, it started to get messier. I learned about REST APIs and Single Page Applications frameworks, but I always felt that it was dirtier than the initial Ruby on Rails simplicity I was introduced to.

I never really dove into the SPA wave; I learned the basics, but for my entrepreneurial venture, Rails was more than enough. Nonetheless, after most of my startups failed, I had the opportunity to work with very talented people who knew the SPA business very deeply. I went through the migration from a Rails monolith to a decoupled Frontend and Backend architecture with them. I would love to say the migration went flawlessly, but I saw how the complexities of the new architecture affected everything from the product itself to the development and engineering process in a small team like ours.

I experienced firsthand the problems with cache when a new release was out and the user did not refresh the page: the slow performance of the first print due to the need to load a big chunk of JavaScript on the first request, the complexities of managing state, the increased difficulty of debugging issues between two codebases and the decrease in deployment speed. At the same time, frontend engineers waited for the backend engineers to perform a change. It was messy.

On the other hand, I was closely following Rails development. I saw how Rails stayed on the sidelines when frameworks like React and Vue started to emerge. Instead, they launched Turbolinks, but that did not go as well as expected. Lots of Jquery code broke. They kept working on it and improving it, and after that, they launched Stimulus JS, a great minimalistic framework that paired beautifully with Turbolinks (I adopted it very early on). But still, if you wanted a very interactive interface, you ended up writing way too much custom JavaScript. But at the end of 2020, things changed. With the release of Hotwire, the puzzle was completed. The best of the Rails framework without sacrificing any interactivity on your apps and all of this with 80% less Javascript writing than usual. Mind-blowing!

So how did they achieve such a fantastic result? The reality is that, as with any great innovation, the process was very iterative. All previous attempts of avoiding Javascript complexity finally paid off. To understand this, let’s try to explain the foundations of the most basic concepts of Hotwire:

Turbo: Turbolinks on Steroids

It is the heart of Hotwire; most of the magic is here. Turbo provides four complementary techniques that help you speed up page changes and form submissions without writing any custom JS, divide complex pages into components, and stream partial page updates over WebSockets.

Everything starts with Turbo Drive (AKA Turbolinks), which is an interceptor for all link clicks and form submissions, that instead of reloading the page, performs requests in the background and then replaces the body (and merges the head) of the document with the returning HTML from the server. This single approach increases the page-level navigation speed a lot because you do not have to reload all the assets, and the speed at which a browser can process HTML is very similar to the speed at which it can process JSON.

For the pages that you do not want to reload completely, you now have Turbo Frames. They work similar to Turbo Drive, but instead of replacing the complete body, you have the ability to encapsulate small parts of the document that can perform individual requests to the server and replace only the content of the matching frame. For example, you can replace an edit button with the form for editing, using the same template for the edit form as if you were visiting the editing page instead. An approach that works great with HTTP/2 and caching.

Turbo Frames are great when we work with direct interactions within a single frame, but what about when we need to update other parts of the pages outside of the Frames? Well, then we can use Turbo Streams. This technique allows us to stream HTML changes to any part of the page in response to updates sent over a WebSocket connection.

These three concepts can take you far, but sooner or later, you will need some customization, and that is what Stimulus JS is for.

Stimulus: A JS framework for the HTML you already have.

Stimulus allows you to connect JavaScript objects to elements on the page using simple annotations. You know that moment when you add a CSS class and the element magically changes position, shape, or color. For me, Stimulus is the same concept but with Javascript instead of CSS. You just need to link your existing HTML with it, and then the magic happens, no query selectors and no need to generate the whole DOM dynamically from Javascript itself. 

The most exciting part of all is that it monitors the DOM for changes. For example, if a Turbo Stream was sent or the content of a Turbo Frame changes, the newly added HTML will connect with its corresponding Stimulus controllers and add the needed functionality immediately. No page reload is needed, and no need to listen for fancy Turbolinks load events.

Conclusion

The development of Hotwire and the enhancements coming to the Ruby on Rails framework are both exciting and refreshing. In a way, it feels like the first time I was introduced to the MVC pattern, a very simple yet powerful approach for building web (and now even mobile) applications without the well-known complexity of building a compelling SPA. 

I encourage anyone reading this, to try these new technologies and rethink the need for complex front-end frameworks. Finally, another path can take us to the same level of interactivity with a fraction of the complexity. It is on us to keep fighting for a world with better frameworks and techniques.

I recommend this approach to you, especially if you have a small development team or you are an indie hacker building stuff on your own. I guarantee you can go 10x faster this way, and you will end up with much better user experiences because of this.

What do you think? Is the web going to be built with more or less JavaScript in the future?

Thanks for reading. Please feel free to share and don’t forget to subscribe to our newsletter below!

In this blog post, we will look at how to solve a simple problem with a Python program, and then we will try to speed it up by using Python’s multiprocessing module.

The main goal of this post is to illustrate how a program can be made much faster by parallelizing work through multiple processes, as opposed to running the whole workload through a single process.

Problem introduction – TCP port scanning

The problem we will be trying to solve is known as TCP port scanning. The problem consists of finding open TCP ports in a given IP address. Such a process could be used by network administrators to identify potential risks in their networks and by attackers to attempt to gain control over exposed systems.

TCP ports are represented in 16 bits, so we have a maximum of 65535 ports per IP address. Port 0 is reserved and cannot be used, so we will focus on the range 1 to 65535.

Given a hostname such as www.google.com, a start-port, and an end-port our program will have to find the IP address of the given hostname and then print all open TCP ports in the given range. 

In order to do this, we will have to make our program iterate through all ports from start-port to end-port and on each step attempt to establish a connection through the current port. If the connection can be successfully established we know the port is open, then we will print a message to let the user know.

A simple single-process solution

Let’s try to create a simple Python function using the socket module. This function takes an IP address and port number as inputs. It returns True if the port is open, and False otherwise.

The function is very simple. First, it wraps its calls inside a try/except block. It then tries to create a connection to the specified address and port. If this connection is successful it will immediately close it and return True, letting us know the port is open. If any problem occurs and the connection cannot be established it will return False, letting us know the port is closed.

The value of timeout=1 is needed to allow our program some time (1 second in this case) to establish the connection. If after 1 second our program can’t establish a connection we will assume the port is closed.

Now let’s wrap our function inside a complete program by reading some command-line arguments and printing appropriate messages. We will use argparse for argument parsing and time to measure execution time.

This is how our program works. When executed it reads the –hostname, –start-port and –end-port arguments. If a port range is not specified it will default to all ports, 1 to 65535. It then creates a variable called start_time to store the current timestamp in seconds, executes the scan_host function, and finally prints the elapsed time in seconds.

The scan_host function first translates the given hostname to an IP address, then iterates through all ports in the specified range and calls our initial is_port_open function for each port. If it finds an open port then it prints a message.

Let’s name our program port_scanner.py and save it.

Scanning 500 ports

Time to do some tests! Let’s see how long it takes to scan through 500 ports.

So our program works just fine and we were able to find two open ports. However, scanning 500 ports took 500 seconds. This is something we could have predicted given our 1-second timeout per connection attempt.

Given this, if we wanted to scan through all 65535 available ports, our program could take 65535 seconds to complete, or a little over 18 hours.

If we don’t have all day to portscan a single host, one thing is clear: our program must run faster.

One thing which comes to mind would be lowering our timeout value, but this could compromise accuracy. TCP connections need some time to establish, and not giving our program enough time could result in wrongly assuming some ports are closed when in reality could simply take some more time to accept a connection.

A better approach to speed up our program would be to try and connect to multiple ports at once, instead of trying a single port at a time. Fortunately, we can achieve this by parallelizing our workload across multiple processes. This is when multiprocessing comes to the rescue.

Speeding things up with Python’s multiprocessing

Python’s multiprocessing module provides a set of classes that allow to spawn subprocesses from a program’s main process. We will look at how we can use the Process class to speed up our port scanning program.

First, let’s modify our scan_host function to take a new workers argument and spawn a set of processes to divide the workload.

Let’s look at the different parts of our new function.

The new argument workers indicate how many subprocesses we want to launch. So given start_port and end_port we can calculate the total number of ports to scan and then divide this number by the amount of workers we will be launching:

At this point, we can iterate through our port range and compute the start_port and end_port of each one of our workers.

So to illustrate this with an example, if we give our program the following inputs:

Then the workers would be set up with the following arguments:

Now in each iteration, we can create a new instance of the Process class to spawn a new subprocess with the given arguments. We will then start the process and store it in our processes list.

When our workers launch they will call the function provided as the target argument of the Process constructor. In this case the function is scan_address. We will look at this function later.

Finally, we will call Process.join on each process to wait until they all finish.

Now, let’s put all pieces together into a new program.

Let’s name our new program port_scanner_parallel.py and save it.

Scanning 500 ports again

Now that we have what should be a much faster port scanner, let’s try scanning 500 ports again. This time we will launch 10 parallel workers.

As we can see, with 10 parallel workers we just gained a 10x improvement in execution time!

Last time, with a single process, it took 500 seconds to scan through 500 ports. Now with 10 parallel subprocesses, it takes only 50 seconds.

Scanning all ports of a host

Now that we have such a fast port scanner we can push things to the limit. Let’s try scanning all 65535 ports of a host with 100 parallel workers.

We were able to scan all ports of this host in just 131 seconds and have found four open ports.

Conclusion

We have looked at how to solve the TCP port scanning problem in Python. We initially looked at a simple single-process solution and then learned how to speed it up by using Python’s multiprocessing module.

We have learned how dividing the workload between a set of parallel workers can offer massive improvements in execution time.

Many computing problems can be parallelized like this, and now that you know how to use multiprocessing you have added a valuable tool to your toolbox. It is now up to you to apply it wisely.

Happy coding!

Thanks for reading. Please feel free to share and don’t forget to subscribe to our newsletter below!