\[ \def\A{\mathrm{{\bf A}}} \def\B{\mathrm{{\bf B}}} \def\C{\mathrm{{\bf C}}} \def\D{\mathrm{{\bf D}}} \def\x{\mathrm{{\bf x}}} \def\y{\mathrm{{\bf y}}} \def\u{\mathrm{{\bf u}}} \def\dif{\mathrm{d}} \def\euler{\mathrm{e}} \]
 

About the Speaker: Aleksei Tepljakov

  • Research Scientist at Centre for Intelligent Systems, Department of Computer Systems, TalTech
  • Head of Re:creation Virtual and Augmented Reality Laboratory, VR First laboratory coordinator
  • IEEE Senior Member, Chair of IEEE Estonia Young Professionals Affinity Group, Vice Chair of IEEE Estonia Education Chapter
  • Primary developer of the FOMCON (“Fractional-order Modeling and Control”) toolbox for MATLAB/Simulink, more information about this project can be obtained from https://fomcon.net/
  • E-mail: aleksei.tepljakov@taltech.ee, website https://atdesign.ee/

Talk outline

  • Introducing the research and development problem
  • Brief review of fractional-order modeling and FOPID control
  • Description of the experimental platform
  • Demo of the working solutions
  • Conclusions and future outlooks

Motivation

  • The rapid pace of technological development clearly necessitates introducing changes into the knowledge transfer model used in educational institutions. At the same time, old teaching methods, including the classic chalkboard or whiteboard instruction style are still relevant.
  • From literature, it becomes apparent that adding interactivity to the classic whiteboard experience can enhance learning outcomes, especially in instruction related to system modeling topics and automatic control design.
  • On the other hand, many additional possibilities that significantly complement the interactive instructions are afforded by the application of extended reality (XR).
  • For example, using the concept of digital twins, one can design a virtual laboratory experience for control system design which is especially helpful if the physical lab becomes inaccessible due to a global health emergency.
  • Next, taking into consideration a typical control systems curriculum in the universities, sufficient coverage of fractional calculus based modeling and control as a generalization of classical modeling and control concepts is still missing from most study programs. Therefore, to improve on this aspect, we consider fractional-order calculus as well.

Interactive whiteboard example

Image source: F. Niebling, D. Schropp, R. Kühn, and T. Schlegel, “Model-based multitouch gesture interaction for diagram editors,” in Human-Computer Interaction. Advanced Interaction Modalities and Techniques. Springer International Publishing, 2014, pp. 121–130.

Alternative: using Logitech VR Ink in XR

Logitech's VR Ink Pilot Edition can be used to develop relevant applications in extended reality.

Research question

Is it feasible to use a physical pen input device for creating an interactive whiteboard experience in extended reality for control system applications including:

  • inserting mathematical models via written input;
  • drawing a desired time domain response of a control loop?

Fractional Calculus

Fractional calculus is a generalization of integration and differentiation to the non-integer order operator $_{a}\mathscr{D}{}_{t}^{\alpha}$, where $a$ and $t$ are the lower and upper bounds of the operation, $\alpha\in\mathbb{R}$ denotes the fractional order such that

\begin{equation} _{a}\mathscr{D}_{t}^{\alpha}=\begin{cases} \frac{\dif^{\alpha}}{\dif t^{\alpha}} & \alpha>0,\\ 1 & \alpha=0,\\ \int_{a}^{t}\left(\dif\tau\right)^{-\alpha} & \alpha<0. \end{cases}\label{eq:FOOP} \end{equation}

If $\alpha$ is an integer number, the definition in (\ref{eq:FOOP}) corresponds to a classical differentiation and integration operation.

FO-FOPDT plants

A FO-FOPDT (fractional-order first-order plus dead time) plant has the following form in the Laplace domain

\begin{equation} G_p(s) = \frac{K}{1+Ts^{\alpha}}\euler^{-Ls} \label{eq:FOFOPDT} \end{equation}

and is characterized by the gain $K$, pseudo time constant $T$, order $\alpha$, and lag $L$. If $\alpha=1$, then this model becomes the classical FOPDT model.

Fractional-order PID controllers

The control law of the PI$^{\lambda}$D$^{\mu}$ controller can be expressed as follows:

\begin{equation} u(t)=K_{p}e(t)+K_{i}\mathscr{D}^{-\lambda}e(t)+K_{d}\mathscr{D}^{\mu}e(t),\label{eq:PIDCtrlAct} \end{equation}

where $e(t)=y_{sp}(t)-y(t)$ is the error signal. After applying the Laplace transform to (\ref{eq:PIDCtrlAct}) while assuming zero initial conditions, the following equation is obtained:

\begin{equation} C(s)=K_{p}+\frac{K_{i}}{s^{\lambda}}+K_{d}s^{\mu}.\label{eq:GCPid} \end{equation}

When taking $\lambda=\mu=1$, the result is the classical integer-order PID controller.

For motivation on using FOPID controllers in the industry, see the recently published paper: A. Tepljakov, B. B. Alagoz, C. Yeroglu, E. A. Gonzalez, S. H. Hosseinnia, E. Petlenkov, A. Ates, and M. Cech, “Towards industrialization of FOPID controllers: A survey on milestones of fractional-order control and pathways for future developments,” IEEE Access, vol. 9, pp. 21016–21042, 2021.

Fractional Control Actions: Integral

Fractional Control Actions: Differential

The contribution

 
 

How it looks like in practice

 

The technical contribution

The interactive whiteboard implementation currently uses the Reveal.js slides framework as basis and the ControlSystems.js framework for several features developed to support the corresponding application, namely:

  • Written input for equations;
  • Drawing the desired time domain response;
  • Plotting all necessary graphs showing time domain and frequency domain characteristics;
  • Module for communicating with an online OCR service for written input conversion into $\LaTeX$ code;
  • Implementation of the $\LaTeX$ parser for processing the response from the external OCR service.

No features related to actual control system functions are used in this application, because the computational burden is offloaded to MATLAB.

MatlabWebSocket

This is used in the present contribution. It is recommended to anyone looking to use MATLAB as a compute node in similar applications where complicated computations must be performed remotely.

Demo Session in XR

 

Conclusions

  • In this work, we have presented a method for implementing an interactive whiteboard in extended reality geared towards solving advanced control design tasks assuming that the classical control curriculum is extended to include the study of fractional calculus in modeling and control.
  • Specifically, we evaluated the use of a tracked physical pen—the VR Ink Pilot Edition from Logitech—in this scenario.
  • Future work should include the improvement of the implementation, revision of the controller tuning strategy to include, e.g., frequency domain characteristics directly.

Answer to the research question

It is definitely feasible to use a physical input device like the Logitech VR Ink Pilot Edition for inserting mathematical equations and drawing the desired signal in an XR environment, but there are a few limitations. Specifically:

  • In this work, we implemented a whiteboard that has no physical counterpart in the real world. While its use is possible, the necessity to provide force feedback simulating the friction of the pen against the virtual whiteboard's surface results in a quick battery drain of the pen.

To improve on this aspect, it is better to use the pad that comes together with VR Ink to provide a physical surface having a virtual counterpart in XR. The location of the pad in XR can be easily tracked using, e.g., and HTC Vive Tracker.

Discussion: Why XR?

 

Discussion: How can students use this?

Visit https://vam-realities.eu/ to download the State-of-the-Art report on XR technology and get more useful information about XR and its applications.

Acknowledgements

  • The work of Aleksei Tepljakov is supported by the Estonian Research Council grant PRG658 and this specific effort was also supported by a teaching grant (with funding source SS19013) awarded to Aleksei by the School of Information Technologies, TalTech, in 2019.
  • We would like to express our gratitude to Logitech for sending a VR Ink Pilot Edition device for evaluation in re:creation XR laboratory, Centre for Intelligent Systems, TalTech.
  • Finally, we would like to thank HTC, who, through the VRFirst (https://vrfirst.com/) program, provided us with an HTC Vive Pro HMD.

Thank you for listening!

Please address further questions to aleksei.tepljakov@taltech.ee