NITHtutorial documentation

Supplementary materials to follow the YouTube tutorial on how to develop simple NITH-based applications

Overview

This repository contains:

• This document, which will repeat the most useful informations (and links) found in the video
• The application described in the tutorial video, already developed, for reference

Pre-requisites to follow the tutorial

The tutorial is designed for Windows operating systems. First of all, ensure the required environment is set up:

• Download and install Microsoft Visual Studio Community
  • While installing, select and include the C# + .NET development tools
• Download and install Python 3.12

Clone the NITHtutorialVideo repository

git clone https://github.com/LIMUNIMI/NITHtutorialVideo

Then launch:

download_materials.bat

download_materials.bat will download:

• NITHwebcamWrapper — turns webcam into a sensor
• NITHtester — to test the NITHwebcamWrapper
• NITHlibrary — core library to handle data
• NITHemulation — secondary library to emulate I/O
• NITHdmis — bonus library to make musical interfaces
• NITHtemplate — blank project to be edited

Launching NITHwebcamWrapper and testing it with NITHtester

Reach the NITHwebcamWrapper folder within a terminal, and run this command to install the requirements (in a virtual environment):

.\setup_env.bat

When finished, run the virtual environment and launch the NITHwebcamWrapper application:

.venv\scripts\activate && python main.py

Let it run in the background. In order to close it, just focus the application window and press ESC.

Then:

• open NITHtester
• set the port to UDP through the selector
• set port number as 20100 if not already set
• click "connect"

If everything works correctly, you should see incoming values visualized. You can try the head tracker visualizer in the bottom of the page.

Following the tutorial video by coding the sample app

An summary of the fundamental steps:

• Open NITHtemplate and modify the Solution/Project name to your preference
• Follow the implementation steps described in the video. If you prefer to copy the code, you will find an already implemented copy in this GitHub repo from which you're reading this readme from.
• Test the application and verify

A list of "exercises" to test your knowledge

After having followed the tutorial, you can try one of the following exercises (tested to be simple) in order to test your skills.

A sip-and-puff clicker

Requirements:

• A specimen of NITHbreathSensor, for which you can find the instructables here.

Try to develop an application that uses a sip-and-puff controller (e.g. NITHbreathSensor) to perform mouse clicks. It could be, for example, that while the user exhales (i.e. puts pressure in the breath sensor), a left mouse button pressure is simulated, which gets released when the exhalation stops.

Some hints:

• An USBreceiver will be needed, connected to the NithModule
• A behavior receiving breath_press will be needed, handling possibly a double threshold strategy to trigger mouse events
• Inside the NITHemulation library, you will find a MouseSender static class, containing SendMouseButtonEvent, from where you can send left mouse button events (check the available flags!)

The head controlled mouse

Requirements:

• NITHwebcamWrapper

Try to develop an application that translates head rotation into mouse cursor movements! Yaw and pitch will control mouse cursor positioning along the X and Y axes.

Some hints:

• An UDPreceiver will be needed, set on port 20100 to receive data from NITHwebcamWrapper running in background, connected to the NithModule
• In NITHemulation there's a NithSensorBehavior_GazeToMouse to take inspiration from ;) which receives gaze data, actually, but it's easily translatable to receive head rotation data.
• You can get screen dimensions like this:

using System.Windows;

// Get screen dimensions
double screenWidth = SystemParameters.PrimaryScreenWidth;
double screenHeight = SystemParameters.PrimaryScreenHeight;

• Then you can declare two SegmentMapper (from NITHlibrary > Tools > Mappers) to map cursor movement, from head rotation to screen coordinates
• Detection will be noisy and messy! You can use a set of three DoubleFilterMAexpDecaying filters to smoothen the signal.

Are you standing still?

Requirements:

• NITHwebcamWrapper

Try to develop a simple GUI interface that turns green if the user is standing still!

Some hints:

• Same as above, plus: there are tools in the library to derive velocities. Inside the folder NITHlibrary > Tools > Mappers you will find a selection of VelocityCalculator classes. VelocityCalculatorTimestamped, VelocityCalculatorDerivative or VelocityCalculatorFiltered may be sufficiently precise for this task.

Wink based slideshow controller

Requirements:

• NITHwebcamWrapper

Try to develop an application that uses left-right eye winks to control the slides in a slideshow (i.e. emulating a left-right arrow key press).

Some hints:

• Same as above for "The head controlled mouse", plus:
• You can add NithPreprocessor_WebcamWrapper to the preprocessor chain in NithModule in order to get boolean values for left and right eyes aperture: they will correspond to NithParameters.eyeLeft_isOpen and NithParameters_eyeRight_isOpen.
• Then, there's an abstract behavior class to extend called ANithBlinkEventBehavior to extend, to handle blinks and winks. The same we used in the tutorial.

Enjoy! But not too much. Life is short :) remember to touch grass, after that.