First look: In/Out, the AI tennis umpire

I wrote back in March about a product called In/Out. The GoPro like device can be fastened to any tennis net post to detect in real-time which side of the line the ball lands. It does this thanks to the same algorithms that provide line-detection for self driving cars.

But that’s not all, In/Out will also detect let serves and provide detailed stats analysis. For each shot, the gadget will track the ball speed, spin and location with 99% accuracy. Further analysis will show shot placement with percentage by zone as seen on TV, calculated for both players.

Essential reading: Tennis gadgets and trackers to improve your game

The $200 gadget has generated quite a bit of buzz in the tennis community and demand for the first batch far outstripped supply. Beeing a bit of a tennis geek and a tech lover, I jumped at the chance to contribute. In fact, I was one of the earliest contributors so was pretty much first in line to receive the gadget.

The wait is now over and, as of a few days ago, I have in posession an In/Out unit. These are my initial impressions.

If you are expecting fancy packaging and detailed instruction booklets, think again. In the mail you get a plain box with an oversized plastic tennis ball and no instructions. Mr Gregoire Gentil, the inventor of the device, perhaps had bigger things to worry about than making sure the packaging looks nice.

Luckily it gets better – in fact it gets much better. What it lacks in presentation, In/Out makes up for in the quality of hardware and software.

In addition to the plastic tennis ball enclosure, you also get several pieces of kit: the In/Out device, L-attachment (along with 4 little feet and band), micro USB cable, battery, and an HDMI micro adapter (donors only). Early contributors also receive a free 8MB microSD card. Thanks Mr Gentil!

What struck me right away was the quality of the main unit. It really looks, feels and operates like an extremely well designed piece of kit. Much more so than I hoped. Lots of work has clearly gone into putting it together. Once you switch it on via the physical button on the right, it feels like you are turning on a small computer. The high-res screen springs to life, and the unit has its own operating system and menus which you can navigate via the touchscreen.

The setup, though, is not as simple as it could be. As there is no instruction booklet you need to head over to this page on the In/Out website which outlines all the steps in detail.

The battery slots in and to start off you need to charge it to its full capacity via the micro USB cable. The red light goes on. When the battery has finished charging, the red led goes off. The battery runs more than two hours in line call mode, and one hour and half when recoding or streaming 1080p video. This can, however, be extended by plugging in a power bank.

Then its time to install the smartphone app and update the firmware on the device. This can be done with the app or alternatively via a PC or a microSD card. I chose the smartphone app method. Rather cleverly, In/Out can act as a hotspot. Download the firmware update to the app, connect the smartphone to the In/Out hotspot and upload the firmware. This is also how you download your tennis sessions from the In/Out device to your smartphone or table. All in all, from start to finish the setup took me about 20-30 minutes.

Then it was time to head over to the tennis court and try it out.

To start using In/Out, you need to secure the L-shaped attachment to a net post. It takes about a minute, so not too much of a problem. The accompanying band stretches around the tennis post and keeps everything in place. Then you need to screw the main unit on top of the L-shaped attachment. Switch In/Out on and you are ready to go.

At this point its worth stressing that while the hardware is the finished product, the software isn’t. In fact, this is still an early beta version of the software. Mind you, it doesn’t feel like a beta version, but it is missing some important functionality. Right now you can only video record your sessions and use the ‘Practice Session’ setting for line calls.

The In/Out Home screen has four options: Line Call: Get automatic line calls (optional video recording); Rec only: Record your play (no line calls); Videos: Playback previously recorded sessions; and Settings: Adjust device settings.

I decided to keep things simple to begin with and try out the video function. One press of a button is all it takes. You can save the video to the internal memory or slot a microSD card in. Once you are done just tap the off button on the screen and the recorded video will appear in the In/Out menu. You can then scroll through all your saved videos and watch them back. However, despite being high res, the In/Out screen is a bit too small to see much detail.

Back home I slotted the microSD card into my computer and uploaded the recording. The video is split into two parts – the top part shows one side of the court, the bottom part the other. This means you get a recording for both players and both sides of the court simultaneously. The quality of the recording is top-notch and it makes it easy to spot the nuances of your form. All in all a very useful function which I can see myself using both in match-play and practice sessions. So far so good.

Right now the stats overlay does not work, but in the coming months your ball speed, spin, location and more will be superimposed on top of the recording. During match play, the gadget will also provide efficient replay – meaning it skips the changeover and towel timeoffs. How cool is that!

Then it was time to test out the line calling system. That was, after all, the big selling point and In/Out’s claim to fame.

The gadget can work on any court – hard court, carpet, clay, grass – as long as there are a few white lines and the lighting conditions are ok.

Before starting play you need to calibrate the court, i.e. give In/Out a chance to detect all the lines. To set up click on ‘Line Call’ making sure the angle of the In/Out device is such so that both sides of the court are in view. You also need to make sure there are no people or balls in the way. Then click on ‘Find Court’. The device will attempt to detect the court lines. If it succeeds, the message “Court found” will appear.

To my frustration, the first time around it was not successful, but then I realised that I did not position the cameras properly. A quick adjustment so that all the lines were visible, and it managed to calibrate the court with no difficulties. I should add here that it had just stopped raining so it was a very dark, cloudy day. Not a problem.

I hit maybe 20-30 balls to test the system out, so it was not a proper tennis session. But it was enough of a test to show me that the system really does work. The unit will flash green when the ball is in, and red when its out. There are also optional sound settings.

Is it perfect when making line calls? No, its not. The system is learning and its still in beta stage. But its already pretty good. It might make the odd mistake, for example, when the ball lands right on the line. On one or two occasions it got confused when I was walking around and called shots in even though I had not even hit a ball!

A very cool feature is that the In/Out screen shows where the ball landed for the last ten shots. And this works flawlessly. You can use the left and right arrows to select a particular shot and click on the screen to see a replay of where the ball landed. The screen zooms in on the ball, just like a TV broadcast when a professional tennis player makes a challenge. Then if there is a dispute, both players can watch the replay and decide whether the ball was in or out.

Think about it. It really is the stuff of magic that the unit is intelligent enough to detect the two players running around on the court, detect the ball flying across the net and landing on the ground, and then decide whether the shot is in or out! And its only going to get better from here as the software is still very much work in progress.

The end goal is for the system to operate with a margin of error of 20-30 millimetres, and its already pretty close to this. While this may not be precise enough for professional tennis players, it is perfectly acceptable for a Sunday tournament match, recreational play or a junior tennis match. Plus there is the replay function which allows players to decide for themselves which side of the line the ball was on.

I will be taking In/Out for more tennis sessions in the coming days and weeks and will play a couple of practice matches. So keep checking back as I will be updating this review. The first practice match will be the real test.

As mentioned, the software is still in beta stage. A future beta is due out before the end of the year and it will have “Match Single” and ‘Serve Training’ modes. Version 1.0 will include the best possible court calibration and best possible line calling accuracy. And version 1.1 will include speed of ball and shots type in the statistics, along with “Match Double” mode. No dates have been set yet in the roadmap for Versions 1.0 and 1.1.

My guess is that it will take another 3-6 months for the software to be 100% complete. But that’s purely a guess. It could be sooner, it could be later.

In my mind I imagine Mr. Gentil as a geniuous inventor, perhaps the equivalent to “Doc” Brown, the inventor of the time machine in the Back to the Future trilogy. But a Palo Alto scientist who is also a tennis geek and has put his skills to resolving frustrations many amateur tennis players face today.

In all honesty I wasn’t sure what to expect as the device did sound like the stuff of science fiction. But I am extremely impressed with the initial result. It is my view that Mr. Gentil loves what he is doing and is building a device that he would want to use himself. And this is why it has so much potential.

I don’t want to sound like a cheerleader, but a gadget that continuously records high-res video of both sides of the tennis court, overlays statistics on top of the recording and even makes line calls with optional video replay – is literally a game changer. The hardware is there but it will be a bit of a wait for the software to catch up. But when it does, don’t be surprised if you start seeing lots of In/Outs strapped to posts of your local tennis courts.

More information can be found on the In/Out website.

Like this article? Subscribe to our monthly newsletter to never miss out!

Leave a Reply

Your email address will not be published.