Firmware and PLC program environment creation of architecture x86

The architecture x86 recently was positioned as embedded and it's real solutions into this industry rarely have resources (< i386), which is not enough for full-featured OS and advanced environment execution. For this reason and by reason of big the architecture unification the individual assemblies of kernel Linux and main programs of the OS environment performed rarely enough, and that typical mostly for the architecture ARM. More interest and practical for x86, for wide the hardware circle, it is assembling firmwares with compressing a root file-system (RFS). But still possible individual assembling by aid of the build systems as "BuildRoot" or "PTXDist", bottom. And also here possible direct installing a Linux distribution.

Instruments for assembling the firmware environments with the compressed RFS, based on distributive ALTLinux

The following requirements were pulled out to the implementation of the PLC firmware of the section:

• Compactness. In connection with the direct dependence of prices on industrial flash drives with their volume, as well as the reality of the needless for frequent updates, the image of the firmware need to be packed, reaching the level of compactness up to the 8-50MB at a runtime PLC in the full-featured OS.
• Uniform source repository. Since the firmware is not something unchangeable, not expandable and finally fixed, then it must be based on the real, developing-supporting repository of OS package. This will allow for a long time to form updates and expansions, not maintaining the mediated repository special.
• Debugged and simple building procedure. Given the fact that configuration of the firmware can be for some time be stabilized and in addition in the operating system's components and in OpenSCADA too it will be detected and eliminated the bugs, the procedure of building of the firmware should not be burdensome, but on the contrary — easily adaptable.
• Clear implementation of the writing mode in the OS tree. Although the firmware means the creation of packaged not modified image of firmware, but the characteristics of the PLC runtime involves the modifying of the databases and archives. In addition, the possibility of correction of the initial configuration is an important requirement.
• Reliability and stability to the sudden shutdowns. The specificity of exploitation of PLC is usually the inability to properly shutdown, as well as the practical reality of the situation instantly and unpredictable power outage. PLC in these situations must keep working, i.e. to contain the journaling file system and ensure that its verification and automatic correction of errors.
• Conditional division of the PLC configuration to two types:
  • PLC without the local display, possibly with a simple text display.
  • Touch-panels with the PLC function.

Given the above requirements, for the creation of the firmware it was chosen the tool for creating the distributions mkimage of ALTLinux. mkimage is the tool for building Sisyphus-based system on basis of template. As an initial set of templates it was taken the set of templates of formation of ALTLinux distributions at git://git.altlinux.org/people/boyarsh/packages/mkimage-profiles-desktop by the command:
git clone git://git.altlinux.org/people/boyarsh/packages/mkimage-profiles-desktop
As the basis it was taken the "rescue" template, as the most compact and close to the target PLC.

Firstly building performed basing on the package base of the distributive ALTLinux 5.1, there is present the realtime kernel from XENOMAI. For obtaining some specific packages you need to connect the repository of packages "ALTLinux 5.1" from the OpenSCADA project:

Assembling

Firstly it was created the configuration of PLC without local display in mind of the availability of this type of equipment and lack of equipment for the Touch-panels.

New PLC template was named "plc", it was tested on the boards of PC/104 form factor MOPSlcdLX of Kontron company, ATH400-128 of Diamond Systems company and modular PLC LP-8781 of the ICP DAS company. The archive of the resulting mkimage tree with the "plc" template can be downloaded here ftp://ftp.oscada.org/OpenSCADA/PLC (templates and materials of individual controllers are placed in their own directories).

The key points of the configuration of new template was the writing of the new init-script (rc.sysinit), the script of the after installation configuration of the firmware's image and the list of packages in the image of firmware.The first script is designed as the package "startup-plc". The second script was embedded in the template "plc" on the way: profiles/pls/image-scripts.d/01system. The list of packages was embedded in the template "plc" on the path: profiles/pkg/lists/plс.in.

The procedure of creating the firmware from the image is the following:
# Creation the configuration script configure $ ./autoconf # Configuring of the builder to generate the disks' images $ ./configure --with-imagetype=flash # Building of the image $ make plc.cd

The result is an output directory in the "profiles/out/" have look:
syslinux/ # boot-loader folder alt0/ # booting files full.cz # image of the primary initialization system vmlinuz # OS Linux kernel syslinux.cfg # configuration file of the loader plс # image of the working OS, OpenSCADA and other tools

Installation

It is possible to download the firmware to: USB-flash, IDE-flash and HDD. However, in the case of the USB-flash there is the problem with waiting for initialization of USB-subsystem and you'll have "to run" some dialogues.

The file system can be FAT or EXT2/3. In the case of EXT3 the root is mounted as in EXT2, because of problems in the initializer. In the case of EXT2/3 you'll need to use not the syslinux boot, but extlinux, the configuration of which is almost the same one.

Next, lets mount the medium and place the files from the output directory on it as follows.
In the case with FAT and syslinux:
syslinux/ # boot-loader "syslinux" folder alt0/ # boot files full.cz # image of the primary initialization system vmlinuz # OS Linux kernel syslinux.cfg # configuration file of the loader plс # compressed image of the root file-system with the OS, OpenSCADA and other tools work # EXT3 file system with the operational data
In the case with EXT2/3 and extlinux:
extlinux/ # directory of the "extlinux" loader (to rename "syslinux") alt0/ # boot files full.cz # image of the primary initialization system vmlinuz # OS Linux kernel extlinux.conf # configuration file of the loader (to rename syslinux.cfg) plс # compressed image of the root file-system with the OS, OpenSCADA and other tools work # EXT3 file system with the operational data

To ensure the reliable operation of the operating data stored in the file "work" with the file system EXT3. The file-system of this file is checked for integrity at the initialization. This file is created as follows:
# Creating the operating data file at 200Mb dd if=/dev/zero of=./work bs=1024 count=204800 # Creating the EXT3 file-system in the file mkfs.ext3 -m 0 work

In the case of the file system EXT2/3 on the target disk the "work" file can not be created. In this case, the working data will be placed in the directory "root" of the target disk.

The next step is the configuration and initialization of the loader. To configure the loader it is necessary to edit the file "syslinux/syslinux.cfg" or "extlinux/extlinux.conf" as follows:
default plс prompt 1 timeout 200 implicit 1 label plс kernel alt0/vmlinuz # To use in the case of identification of the bootable partition by the label append initrd=alt0/full.cz live lowmem fastboot stagename=plс showopts automatic=method:disk,label:PLC # To use in case of identification of the bootable partition by the identifier # append initrd=alt0/full.cz live lowmem fastboot stagename=plc showopts automatic=method:disk,uuid:4824-271E

In the case of selection the identification of the bootable partition by the identifier you can get the ID of our partition with the command: blkid.

In the case of the label it is a bit harder and this is done for different file systems in different ways.

For the file-systems EXT2/3 it is done by the utility e2label. For example: e2label /dev/sdb1 PLC

For the FAT file system it does by the set of utilities that come with mtools or with parted, easier. With mtools you can do it as follows:
# Edit /etc/mtools.conf by the line addition: drive c: file="/dev/sdb1" # Where /dev/sdb1 is bootable partition of the target disk # Setting the partition's label mlabel c:PLC

Now we can initialize the loader:
# For "syslinux" syslinux /dev/sdb1 #previously the partition must be unmounted # For "extlinux" (path points to mount point of the partition of the target disk /media/PLC) extlinux --install /media/PLC/extlinux

That is all with the boot and initialization of firmware. If the resulting disk is not loaded:

• The bootable flag is missing at the bootable partition.
• The incorrect or damaged "MBR". To check and restore it is possible by usage of "ms-sys" utility.
• Sections of the media are created or recreated with the help of parted. This utility align in strange way the partitions that do not allow them to boot on USB-flash. It is necessary to repartition the media with fdisk.
• Boot can also be not working on the old systems that do not know how to boot from USB-HDD. For them it will be necessary to adapt the geometry of the disk being arranged to USB-ZIP or something like that.

Result

The result is the firmware with the size from 30Mb to 100Mb, satisfying all announced requirements and it provides:

• Loading for 27 seconds from the controller's switch on and including the initialization of BIOS.
• Checking and restoration of the working file system in the "work" file.
• Storing the user data and changes of the firmware in the "work" file.
• Automatic network configuration with DHCP (or 192.168.0.1).
• Access to the controller via SSH and user-friendly interface of working, including mc. The passwords by default (root:123456; admin:123456).
• Time synchronization via NTP.
• OpenSCADA execution with the available network interfaces:
  • configuration and runtime environment via Web (ports: 10002 and 10004);
  • control interface of OpenSCADA (10005).

OpenSCADA

As the PLC runtime system the OpenSCADA is used. For this case we'll take the building with separate packages for each module and indicate to install the virtual package openscada-plc, which contains all the dependencies on all the OpenSCADA packages, typical used for this configuration. The package of gd2 graphics library has been rebuilt without the support of xpm graphic file format and library was called libgd2-noxpm. All this was done in order to avoid the heavy dependencies on the libraries of GUI XOrg.

The result is the runtime of the PLC with support:

• DB:
  • SQLite.
• Transport:
  • serial interfaces;
  • TCP, UDP and UNIX sockets;
  • security socket layer (SSL).
• Transport protocols:
  • HTTP;
  • self control protocol of OpenSCADA;
  • user protocol.
• Data source:
  • calculator on the Java-like language of high-level;
  • calculator on the functional blocks;
  • controllers of the logical level;
  • various PLC by the protocol ModBus (RTU,ASCII,TCP);
  • OS data;
  • hardware of "ICP DAS", for controllers LP8x81.
• Archiving:
  • on the file-system.
• User Interface:
  • Visual Control Area (VCA) engine;
  • visualizer VCA based on the Web-technologies;
  • configurator of OpenSCADA based on the Web-technologies.

The configuration of OpenSCADA runs in demon mode in locale "en_US.UTF-8" (also available "ru_RU.UTF-8" and "uk_UA.UTF-8") using the local database SQLite, providing the following default network services:

• configuration and execution environment through the Web (ports: 10002 and 10004);
• control interface of OpenSCADA (port: 10005).

The implementation details

In this section let examine the details of the OS tree of the firmware, the initialization script rc.sysinit.plc and the script of preparation of the OS tree of the firmware.

To build the PLC firmware it was used the following list of packages:
interactivesystem ### Startup kernel-image-@KERNEL@ rootfiles startup startup-plc SysVinit hwclock ### Common coreutils glibc-locales glibc-nss glibc-utils glibc-timezones sysfsutils sysklogd util-linux schedutils ### Disk utils hdparm ### Applications binutils pciutils procps shadow-suite ### Applications/Archiving gzip ### Applications/File less ### Filesystem utils e2fsprogs ### Applications/Networking etcnet iputils mailx openssh-server dhcpcd ntpd ### Applications/Shells bash mc udev dbus hal libgd2-noxpm fonts-ttf-liberation lm_sensors nut nut-driver nut-server watchdog ### OpenSCADA Console #openscada-plc ## OpenSCADA GUI openscada-visStation autologin icewm wm-select xorg-x11-server #xorg-drv-geode xorg-x11-utils fonts-ttf-dejavu fonts-bitmap-terminus

List of the modules of the loader's system kernel with the purpose to reducing the initialization image size was decreased to the following ones:
loop.ko mtouch.ko pcmcia_core.ko rsrc_nonstatic.ko yenta_socket.ko scsi_mod.ko libusual.ko ide-disk.ko ide-core.ko sd_mod.ko usbcore.ko ehci-hcd.ko ohci-hcd.ko uhci-hcd.ko appletouch.ko usbhid.ko usbtest.ko usb-storage.ko mbcache.ko jbd.ko ext2.ko ext3.ko fat.ko nls_base.ko nls_cp866.ko nls_koi8-r.ko nls_utf8.ko zlib_inflate.ko squashfs.ko unionfs.ko vfat.ko pata_cs5536.ko amd74xx.ko

To the script of the tree preparation there were added the following functions:

• changing the name of the distribution kit;
• replacing the init-script to "inittab.plc";
• creating the user-admin "admin" and the passwords setting by default to "123456" for users "root" and "admin";
• initialization of the /etc/fstab;
• setting locale to "en_US.UTF-8";
• network configuration;
• clock setting on the whole and the time-zone setting to "/usr/share/zoneinfo/Europe/Kiev", for changing what you need replace the file "/etc/localtime" to need zone;
• enabling of the necessary services;
• removing of the documentations, help pages and information, icons, RPM-database and apt cache;
• removing of the not needed locales and the translations; left for presence only the locales: en_US, uk_UA and ru_RU;
• selection of only used kernel modules and deletion all the others is added; it is disabled by default and can be enabled to preparation the final firmware for specific equipment;
• the directory with the kernel (/ boot) is deleted in connection with its moving to the root of the boot partition.

The initialization script (rc.sysinit.plc) was provided with the following functions:

• remounting the root partition to the RW mode;
• checking and connection of the file system (/image/root) of the working user data (the file "work");
• reflection of the modifiable directories of the PLC tree (/etc, /var, /root, /home, /mnt and /lib) to the file system with the user's working data (/image/root) through the file system "aufs" or "unionfs".

As the result of these actions the mount table of the resulting PLC tree looks like:
[root@localhost ~]# cat /etc/mtab rootfs / rootfs rw 0 0 udev /dev tmpfs rw,size=10240k,mode=755 0 0 /dev/hda1 /image vfat rw,fmask=0022,dmask=0022,codepage=cp866,iocharset=utf8,check=r 0 0 /dev/loop0 / squashfs ro 0 0 /proc /proc proc rw 0 0 sysfs /sys sysfs rw 0 0 tmpfs /tmp tmpfs rw 0 0 /dev/loop1 /image/root ext3 rw,noatime,nodiratime,errors=continue,data=ordered 0 0 /image/root/etc /etc unionfs rw,dirs=/image/root/etc=rw:/etc=ro 0 0 /image/root/var /var unionfs rw,dirs=/image/root/var=rw:/var=ro 0 0 /image/root/root /root unionfs rw,dirs=/image/root/root=rw:/root=ro 0 0 /image/root/home /home unionfs rw,dirs=/image/root/home=rw:/home=ro 0 0 /image/root/mnt /mnt unionfs rw,dirs=/image/root/mnt=rw:/mnt=ro 0 0 /image/root/lib /lib unionfs rw,dirs=/image/root/lib=rw:/lib=ro 0 0 devpts /dev/pts devpts rw,nosuid,noexec,gid=5,mode=620 0 0 shmfs /dev/shm tmpfs rw,nosuid 0 0

Setting of the GUI

One option of the firmware is built with a graphical interface, which, however, necessary to configure for automatic startup with the visualization area of OpenSCADA. In addition, it should be noted that the firmware with a graphical interface does not contain all the drivers and you may have to rebuild it under the right equipment.

After downloading and logging to the console it is necessary to configure the XServer, automatic graphical login, start of the graphical environment and automatic startup of OpenSCADA from the IceWM environment:
# XOrg configuration file downloading from ftp://ftp.oscada.org/OpenSCADA/PLC/xorg.conf, # placing it to folder /etc/X11 # !! File /etc/X11/xorg.conf edit to self needs. # Keyboard's layout switch adding for: English, Russian, Ukrainian $ echo "-option grp:lctrl_lshift_toggle -variant ,winkeys,winkeys -layout us,ru,ua -model pc104" > /etc/X11/xinit/Xkbmap # Test startup $ startx # Configuration of an automatic graphical login by creating the file /etc/sysconfig/autologin with the contents: $ echo "AUTOLOGIN=yes" > /etc/sysconfig/autologin $ echo "USER=admin" >> /etc/sysconfig/autologin $ echo "EXEC=/usr/bin/startx" >> /etc/sysconfig/autologin # Enabling, startup of the graphics at boot $ chkconfig dm on # Starting the graphics subsystem $ service dm start # !! Further actions are made in the terminal from the graphical interface # Creating the configuration file of automatic OpenSCADA startup from the IceWM $ mkdir ~/.icewm $ echo "#!/bin/sh" > ~/.icewm/startup $ echo "xset -dpms; xset s off" >> ~/.icewm/startup $ echo "/usr/bin/openscada_start" >> ~/.icewm/startup

Packages base of ALTLinux T6

Next stage of the firmwares creation was moving to the package base of distributive ALTLinux T6. On the whole firmwares creation concept saved, with bits changes, but there were added some improvements and expansions:

• Used the new function of "propagator" (the system of pre-initiation of hardware, searching and starting to the firmwares on FS) for searching/creation a partition of EXT2/3/4 with the label "alt-live-storage" for the "root" partition forming and it's modification possibility. The function provides a feature of installing packages direct from the distribution repository, and updating packages packed to the firmware, exclude the kernel and some system's services.
• Added the possibility of creation the firmware as combined ISO-image, which you can (besides it writing to CD/DVD) direct, at aid of the utility dd, write to USB-flash, HDD, SSD and get a work environment with the partition "alt-live-storage", the "root" reflection, on the storage's free place.
• For the possibility to creation of new firmwares to "LP-8x81" from ICP DAS was done moving the real-time kernel "rt-up" from the repository "ALTLinux 5.1" to "ALTLinux T6". The kernel "rt-up" was successfully adapted and get the working firmware based on the packages base "T6" for "LP-8x81".

During the possibility of a free additional installation of needs packages direct from the repository gone needs to the separated built of the firmware with GUI. That is you can easy install the desired window manager (WM) or desktop environment include needed drivers, than create a separated firmware with a limited list of the drivers.

The script "startup-plc" was turned spare into the new firmwares besides the "root" FS remounting to writing does early on the initial sage. The script "profiles/plс/image-scripts.d/01system" renamed to "profiles/plс/image-scripts.d/init1-PLC", but it changed and expanded. The packages list of the firmware was left into "profiles/pkg/lists/plс.in" and some changed.

For get the some specific packages you have to connect the repository "ALTLinux T6" from the OpenSCADA project:

The firmware creation procedure mostly left unchanged:


The output folder's content with the image and the firmware installing process to file system FAT and EXT2/3/4 different only by renaming the FS archive's file from "plc" to "live". Installing the ISO-image to USB-flash, HDD, SSD performs by the command dd:

Instead the file "work" you should create partition EXT3 with the label "alt-live-storage", if it is not the ISO-image. The new partition creation you can do with the help of fdisk, if the FAT partition was not created to all allowed the storage space, or with help of parted where the FAT partition you allowed to change. To the details about a partition creation the reader will send to the documentation on fdisk or parted.

Configuration of files "syslinux/syslinux.cfg" and "extlinux/extlinux.conf" were not changed, besides the FS file archive's name changed from "plc" to "live".

At the result we get the firmware with size from which provides:

• Loading for 25 seconds from the controller's switch on and including the initialization of BIOS.
• Checking and restoration of the journal of the working file system "root" into "alt-live-storage".
• Storing the user data and changes of the firmware, also new and updated packages, in the partition "alt-live-storage".
• Automatic network configuration with DHCP (or 192.168.0.1), for the first interface.
• Access to the controller via SSH and user-friendly interface of working, including mc. The passwords by default omit and the user "root" allowed for connection to it via SSH.
• Time synchronization via NTP.
• OpenSCADA execution with the available network interfaces:
  • configuration and runtime environment via Web (ports: 10002 and 10004);
  • control interface of OpenSCADA (10005).

For the PLC firmware assembling used next packages list:
# INIT 3 # acl usbutils screen acpid acpi anacron vim-minimal mc sound-scripts alsa-utils apt udev udev-initramfs dbus schedutils pciutils setserial lm_sensors3 rsync interactivesystem su system-report mtools netcat strace binutils syslogd glibc-utils glibc-gconv-modules glibc-nss glibc-timezones glibc-locales shadow-utils keyutils lsof sudo # INIT3: BLOCK DEVICES # hdparm fdisk ms-sys syslinux dosfstools e2fsprogs #network etcnet dhcpcd xinetd iftop lftp wget iptables ntp ntpd ntpdate tcpdump multipath-tools openssh-clients openssh-server ppp-pppoe #### LP kernel-modules-icp-@KERNEL_MOD@ #### Console OpenSCADA libgd2-noxpm openscada-plc openscada-DAQ.DiamondBoards openscada-DAQ.ICP_DAS openscada-DAQ.Comedi

The Linux kernel modules list of the initial stage was some changed and include:
loop.ko mtouch.ko pcmcia_core.ko rsrc_nonstatic.ko yenta_socket.ko scsi_mod.ko libusual.ko ide-disk.ko ide-core.ko ide-gd_mod.ko sd_mod.ko usbcore.ko ehci-hcd.ko ohci-hcd.ko uhci-hcd.ko appletouch.ko usbhid.ko usbtest.ko usb-storage.ko mbcache.ko jbd.ko ext2.ko ext3.ko fat.ko nls_base.ko nls_cp866.ko nls_koi8-r.ko nls_utf8.ko zlib_inflate.ko squashfs.ko unionfs.ko aufs.ko vfat.ko pata_via.ko pata_cs5536.ko amd74xx.ko

To the script of the tree preparation "profiles/plс/image-scripts.d/init1-PLC" performs the functions:

• changing the name of the distribution kit;
• enabling login from the user "root" via SSH, without a password;
• setting locale to "en_US.UTF-8";
• network configuration for use DHCP or set to "192.168.0.1" for the first interface;
• clock setting on the whole and the time-zone setting to "/usr/share/zoneinfo/Europe/Kiev", for changing what you need replace the file "/etc/localtime" to need zone;
• enabling of the necessary services;
• removing of the documentations, help pages and information, icons;
• removing of the not needed locales and the translations; left for presence only the locales: en_US, uk_UA and ru_RU;
• selection of only used kernel modules and deletion all the others is added; it is disabled by default and can be enabled to preparation the final firmware for specific equipment; it was unified for the modules list setting by groups of the kernel subsystem’s and specified for hardware;
• the directory with the kernel (/ boot) is deleted in connection with its moving to the root of the boot partition.

Real-time kernel

For a series of tasks are important, often also critical, criteria of the environment is the real-time handing level, then it is possibility of working the tasks according to the real-time priorities and provision of a reaction to events by that priorities.

The Linux kernel of itself provides POSIX real-time scheduling policies "SCHED_FIFO" and "SCHED_RR" with the priorities range (0...100). But the important criteria is "Timer frequency and the reaction to it" up to version of Linux kernel 2.6.24 was too low, for criteria of the real-time systems. In modern kernels of Linux (> 2.6.24) provided support for timers of the high-precision resolution (HPET), what decreased the reaction time to a timer up to 100 microseconds, but that time stability is not guaranteed. To ensure stability of reaction to a timer on level 60 microseconds, and also series of the other criteria of real-time, at the moment you need to assemble a kernel with one real-time extension.

On the ALTLinux distributions observed the kernel 2.6.29-rt-up, which assembled with the real-time extension of real-time is XENOMAI. Into other distributions, for example OpenSuSE, observed also the solutions with it extension.

For now higher criteria of the real-time ensured of the extension The Real Time Preempt Patch, on enabling here full it's features by (CONFIG_PREEMPT_RT), The patch assembling to Linux kernels process and it's work result will trace into this section.

For the real-time level testing on different kernels will use the utility "Cyclictest", which typical call command line and it's arguments looks: "$ cyclictest -t1 -c1 -p 80 -n -i 200 -l 100000". Where:

• -t1 — single testing thread;
• -с1 — using the clock of real-time CLOCK_REALTIME;
• -p 80 — priority of the testing thread;
• -n — using the function "clock_nanosleep";
• -i 200 — pulling interval of the testing thread: 200 us, as closer to the average value 50 us;
• -l 100000 — testing iterations number.

Pair of measurements for Linux kernels of the wide purpose:

• ALTLinux T6 distribution standard kernel (3.0.79-std-def, LP8781):
  • loading 0%: Avg: 37 us; Max: 152 us
  • loading 100%: Avg: 37 us; Max: 191 us
• ALTLinux T6 distribution kernel (3.4.45-un-def, LP8781):
  • loading 0%: Avg: 53 us; Max: 217 us
  • loading 100%: Avg: 40 us; Max: 183 us

kernel-image-rt-up-2.6.29

The kernel originally presents into the distribution ALTLinux 5.1, and it is also moved to the local repository of the project OpenSCADA for ALTLinux T6. The kernel assembled with the extension XENOMAI and AUFS, allowing it using into the firmwares and packed root FS, that was done for the PLC LP-8x81.

Results of the kernel testings:

• LP8781 (XENOMAI)
  • loading 0%: Avg: 26 us; Max: 136 us (there observed jumps up to 1 ms, like to instability the source clock "tsc", at that the "acpi_pm" worse here)
  • loading 100%: Avg: 24 us; Max: 3655 us
• LP8781 (CONFIG_PREEMPT_NONE)
  • loading 0%: Avg: 29 us; Max: 71 us
  • loading 100%: Avg: 28 us; Max: 117 us (there observed jumps to continuous the measured value rising)
• LP8781 (CONFIG_PREEMPT_RT)
  • loading 0%: Avg: 29 us; Max: 64 us
  • loading 100%: Avg: 29 us; Max: 82 us
• AMD Turion Neo X2 L625 (CONFIG_PREEMPT_RT)
  • loading 0%: Avg: 57 us; Max: 74 us
  • loading 100%: Avg: 57 us; Max: 78 us

As you can see from the testing results the patch XENOMAI does not ensure proper level of the real-time on using the standard mechanisms of the POSIX real-time scheduling, at the same time as the kernel version 3 even without the specific real-time extensions ensures the clearly better result.

Necessity for assembling same that kernel with the patch/parameter CONFIG_PREEMPT_RT is actual by presence a number of binary modules from ICP_DAS, for "LP8x81". Also here is actual a question of building the kernel 2.6.33 at the same reasons but for "LP8x81 Atom". The preliminary assemblies of the kernels 2.6.29 and 2.6.33 reveals series of problems which will described here. Solving also a variant to build a modern kernel with CONFIG_PREEMPT_RT and next to request for building these binary modules from "ICP DAS".

Assembling and testing process:

1. Patches CONFIG_PREEMPT_RT and AUFS of days of 2.6.29 are conflict on the function "debug_mutex_set_owner()", into CONFIG_PREEMPT_RT it removed — replaced to "mutex_set_owner()".
2. On the assembling here detected series of problems for "# typedef void irqreturn_t;" — replaced to "#include <linux/irqreturn.h>".
3. The first start with CONFIG_PREEMPT_RT, but without AUFS, was successful — the results above.
4. The starting with AUFS was reveal a problem into memory allocation by AUFS into "aufs_mmap()" — the working code of AUFS was taken from the preliminary assembling "rt-up-2.6.29.alt2".
5. It starting with AUFS was reveal a problem of hanging on the FS root into AUFS, like to possibility of cycling/blocking a RT-task — setup CONFIG_PREEMPT_NONE, on LP8781 and "AMD Turion" any problem does not observed (possible the problem due the HPET missing on PLX8).
6. At the first look the kernel works fine, but observed jump to continuous growing the measured value of the delay time.
7. There finished adapting of the kernel to a binary compatibility for the modules "slot" and "icp" from ICP_DAS. The module "8250_linpac" crashes at it's loading, and "icpdas_8250" has more unresolved symbols — the modules need to reassemble or to try the interfaces > COM2 initiate through setserial — the modules was reassembled by the help of Golden Wang (technical support of ICPDAS).
8. The new kernel set to high loading by the project configuration ACS ball drum mills:
   • network with the driver "via_rhine" halted, after 4 days of successful working — the halt expected, assembled the driver "rhinefet", testing continued.
   • on the driver "rhinefet" the system on loading had work three weeks. But here observed the interrupt 11, on what hangs mostly all standard hardware (USB, Ethernet and may be something), had disabled and the network continued to work into "Pool" mode, which is slower. Possible that interruption disabling occurs also with "via_rhine", and it can not work in the mode "Pool", why packages into the network do not go. The problem reason linked to halt and generation the unhandled interruptions from one of hardware on the interruption 11.
   • The problem fixed by preventing the interruption disabling at help the Linux kernel parameter "noirqdebug".
   • The adapting successful finished and the firmwares based on the kernel ready to the production implementation!
   • 01.03.2015: Instead function EnableWDT() used EnableSysWDT(), by limit to 30 seconds and cyclic reloading if the system was not loaded in 30 seconds (up to three reloadings).
   • 17.03.2015: With assist the ICP_DAS support service was fixed a problem into the serial interfaces driver for more to COM2 which causes to Linux kernel "freeze" (like to interruptions block) after closing one port and activity on some other.
   • 29.07.2015: Detected one more problem looks like by the symptoms to the interruption 11 disabling, but: the interruption 11 is not disabled and all other devices on it works. It reproduced only on configurations with using that both network interfaces, at that possible "braking" for one of its. The problem resolved only by reloading "the braked" network interface, by the command: ifdown eth0; ifup eth0. To detect it and the reloading performs we recommend on the OpenSCADA level append the traffic control and same reloading of the interface on the traffic lack.
   • 21.11.2016: Driver "rhinefet" has been adapted to prevent the interrupts lock and the interrupt vector disable but SHARE mode using. For now it works but 19.12.2016 also there was observer the two adapters network slowing after about two week working.
   Then the hardware is broken for two adapters work and for the PLC you can use only one for stable work!

Result kernel, which renamed to "kernel-image-rt1-up-2.6.29.alt1", you can use for PLC with HPET or a high precision timer, and also into "LP-8x81" and "LP-8x81 Atom" (only single kernel)!

kernel-image-rt-up-2.6.33

Kernel version 2.6.33 assembling with the patch CONFIG_PREEMPT_RT needs for PLC LP-8x81 of firm "ICP DAS" and LP-8x81 Atom (main and original kernel) by reason of presence for it the binary drivers of "ICP DAS".

Testings results of the kernel:

• AMD Turion Neo X2 L625
  • loading 0%: Avg: 65 us; Max: 86 us
  • loading 100%: Avg: 57 us; Max: 72 us
• LP8781
  • loading 0%: Avg: 37 us; Max: 88 us
  • loading 100%: Avg: 34 us; Max: 108 us
• LP8781-Atom (original assembling of the kernel 2.6.33.7-rt29-ICPDAS)
  • loading 0%: Avg: 17 us; Max: 50 us
  • loading 100%: Avg: 12 us; Max: 32 us

Assembling and testing process:

1. Assembling of the kernel from sources of "ICP DAS" (2.6.33.7) and configuration it inherited from kernel 2.6.29 (the source code has suspiciously more of *.rej files, and also "staging/comedi" impossible to build) — it started and mostly works; modules "ipic" and "slot" loaded; module "8250_linpac" crashes into function "platform_device_add"; series of programs hangs into FS operations, with the message: "task openscada:2153 blocked for more than 120 seconds".
2. AUFS replacing to the version from 2.6.29-rt1 — crashes into rtmutex on starting; replacing to official version from git looks same result, at the begin used patch "aufs+sqfs4lzma-2.6.33.patch" from DLink.
3. Assembling the original kernel with patches CONFIG_PREEMPT and AUFS — a problem again into AUFS, but now it at the finish can not look "/sbin/mingetty".
4. Assembling the original kernel 2.6.33.9 with patches CONFIG_PREEMPT and AUFS — same problems.
5. Assembling from sources of "ICP DAS" (2.6.33.7) for SMP — the module DAQ.JavaLikeCalc of OpenSCADA crashes at an unknown reason.
6. Assembling the original kernel with patches CONFIG_PREEMPT and AUFS for SMP — same problem as without SMP but it is not immediately and about five thread.

Diamond Systems ATH400, DMM32

The boards from "Diamond Systems" were a first on which OpenSCADA had tested and stabilized. First board here was single-board computer ATH400 of the form-factor PC/104, shown on the image bottom. On the computer a OS environment early was installing in traditional method and for now by the presented conception of the firmwares assembling.

Hardware specification of the board:

Central processor:	VIA Eden 400-660MHz
Operational memory:	128MB, soldered to the board
Permanent memory:	IDE (44) port with UDMA-33
Video subsystem:	S3 Savage 4 Chipset with extension of 3D/2D video; supported flat panels, CRT and LCD
Audio subsystem:	present
Interfaces/ports:	10/100Mbps Ethernet; 4 RS-232 serial ports; 4 USB 1.1 ports; PS/2 keyboard and mouse
Power supply:	+5VDC ±10% @ 2A
Operating conditions:	-40 ... +85°C
DAQ:	16AI (16 bit, 100 kHz), 4AO, 24DIO, 2CNTR

First task of the board and OpenSCADA was implementation of function of high-frequented (10 kHz * 8 channels) signals archiving of a gas compressor for learning the surge phenomenon on Anastasievskaja GLKS.

Second task was the PLC prototype of power substations nodes. Already for this task the firmware had assembled on base of new package base. Into the task Data acquisition module of boards of the "Diamon Boards" was unified for support all boards of the manufacturer.



Second board was one board of the interface to the object DMM32-AT, shown on the image bottom. The board used for creation of mockups and emulators of field of technological programs. The board has specification:

Power supply:	+5VDC ±10% @ 200mA
Operating conditions:	-40 to +85°C
DAQ:	32AI (16 bit, 200 kHz), 4AO, 24DIO, 2CNTR

Analysis of noise properties of high-impedance inputs of boards from Diamond Systems in different conditions placed to table bottom:

Conditions	External PS: Noise level, mV [Hz]	Internal PS: Noise level, mV [Hz]	Notes
(10 kHz, ±0.625 V, 20 us), Grounded-32	0.08 [-]
(10 kHz, ±5 V, 20 us), Grounded-32	0.5 [-]	0.35 [-]
(10 kHz, ±5 V, 20 us), Grounded-31, Loading ∞	5 [50, 270]	18 [50, 150]
(10 kHz, ±5 V, 20 us), Grounded-31, Loading 1mOm	10 [50, 270]
(10 kHz, ±5 V, 20 us), Grounded-31, Loading 100kOm	7 [150, 50, 270]	7 [-]
(10 kHz, ±5 V, 20 us), Grounded-31, Loading 10kOm	4.5 [100]	6 [-]
(10 kHz, ±5 V, 20 us), Grounded-31, Loading 1kOm	0.9 [150]	1 [-]
(10 kHz, ±5 V, 20 us), Grounded-31, Loading 100Om	0.5 [-]	0.5 [-]

Kontron MOPSlcdLX

MOPSlcdLX is single-board computer of form-factor PC/104, shown on the image bottom. The computer had attracted attention by some low price, by ordinal conditions of the exploitation and then lower heating.

Hardware specification of the board:

Central processor:	AMD LX800™ 500 MHz, fanless
Operational memory:	DDR-RAM-SODIMM socket
Permanent memory:	IDE (44) port
Video subsystem:	built-in graphic with support output to flat panels, CRT and LCD
Audio subsystem:	no
Interfaces/ports:	10/100Mbps Ethernet (Intel® 82551ER); 2 RS-232 serial ports; 2x USB 2.0; PS/2 keyboard and mouse, Floppy, LPT
Poser supply:	+5VDC ±10% @ 1A
Operating conditions:	0 ... 60°C

The board had used for testing of creation operator stations/panels with OpenSCADA into like environment. Latter there had occur problems series with the board:

• The clock's battery lack then it needs to setting the time at any starting or setting the external battery. At that the BIOS settings stored not in CMOS but in FLASH, it is kind good but if the BIOS configuration reset impossible then it is too bad.
• A problem into the interruption scheme which cause to block all or mostly adjacent interruptions at the module "geode_aes" loading.
• In some time the video-card or it's video-output burned but the configurations of BIOS stored into FLASH and there impossible to return its, for fix possible disabled the video into the BIOS. Also the BIOS do not support of forwarding self interface to a COM-port. Will try use CRT here, when it emerged.
• The high-precision counter (HPET) lack, that surprisingly and make this board unusable for real-time systems but ordinal counter can provide lag about 4ms.

For now base on this board preparing "The program oscilloscope" why it was successfully downloaded by the modern firmware with kernel "std-def" and addition board DMM-32X-AT installed, about that see above.

Tri-M VSX104

VSX104 is single-board computer of form-factor PC/104, sown in the image below. The computer had attracted attention by the reason of low price and low power consumption (< 2W) and followed low heating. But by using the processor "Vortex86SX-300 МГц" here needs specific points to forming of the OS environment, but here used CPU instructions set of i486 and the mathematical coprocessor lack.

Hardware specification of the board:

Central processor:	DM&P SoC Vortex86SX-300 MHz
Operational memory:	128 МБ DDR2 RAM, soldered to the board
Permanent memory:	slot CompactFlash Type I, microSD and 2 ports EIDE (Ultra DMA 100)
Video subsystem:	no
Audio subsystem:	no
Interfaces/ports:	10/100Mbps Ethernet; 4 RS-232 serial ports; 2x USB 2.0; PS/2 keyboard and mouse, LPT, Redundancy, SPI
Power supply:	+5VDC ±10% @ 370mA
Operating conditions:	-40 ... 85°C
DAQ:	1 port 16-bit GPIO

For now there is not a solution based on the board but planed to assemble for it a environment, possible for an autonomous control resources system.

ICOP VDX-6354D

VDX-6354D is single-board computer of form-factor PC/104, sown in the image below. The computer builds on the processor "DM&P SoC CPU Vortex86DX- 800MHz", which more productive to "Vortex86SX-300 МГц" into above section, but the power consumption for this board already not 2 but 4 Watts see details on the project page (RU).

ICP DAS LP-8x81

Industrial controllers of family LP-8x81 of firm ICP DAS are a first product from series LinPAC built on x86 compatibility processor, early created controllers of this family based on processors of ARM family. Besides the x86 processor these controllers have significant resources of the operative memory and main storage's space. All that had allowed see to the controller as first candidate from products of ICP DAS for OpenSCADA adapting as environment of execution see details on the project page.

Avalue Sensor panel FPC-1701

Firm Avalue in accordance with Russian firm ElTech provide wide range of panel PC which you can use starting from ordinal mono-blocks and ending by industrial sensor panels into the frontal protection class IP65. By a standard hardware using here you can start and work in OpenSCADA without a problem see details on the project page (RU).

iROBO-3000a

iROBO-3000a is a fanless industrial computer with Intel Atom D425 1.8 GHz with VGA, 2xGb LAN, 4xCOM, 4xUSB, 1GB RAM, 1x2.5" SATA HDD 120GB, Mini-PCIe, 4x4 DIO, CF slot, SIM Card slot, Audio, WDT on board, operating temperature range -5...+55°C. Performance of this computer is enough to run the functions of data acquisition, monitoring and control server, as well as the visualization station's functions. However, because of usage the non-productive Atom processor family, the implementation of mathematical models of processes will require almost all of the CPU resources. For example, during the performance of the AGLKS mathematical model, the CPU is loaded at 86%. The controller has been certified by "UKRSEPRO" that may be important for many users in the territory of Ukraine.



OpenSCADA operating environment for this computer was based on the packets base of the ALTLinux T6 distribution, as well as freshly-builded Trinity (TDE) desktop environment. Building of the environment was made using the above described conception with an updated profile of "mkimage". The "plc" objective has been added to the new profile, but its nature has changed in fact and has become a copy of the "live" target, which became possible thanks to the implementation in primary initialization stage the transparent mount of the partition with the "alt-live-storage" label as a reflection of a packed file system with random access to the modification. In general, it made possible to create the fixed core of the firmware with the basic set of software environment with the size of 300MB and with the possibility of free expansion by installing the necessary packages from the distribution.

The Trinity was selected as the desktop environment because of the presence of background artefacts problem in conjunction with XOrgServer 1.10 + Qt4, as well as because of TDE low-resource with high maturity and stability.

Archive of the build profiles of the new environment is called mkimage-profiles-6-kdesktop-plc.tgz, and the latest build of the firmware ALTLinux_6-OpenSCADA_0.8.1-TDE_3.5.13.1-i586-flash.tar.

Advantech PCA-6753, PPC-L126

Firm Advantech produces wide range of hardware for automation, start from panel PC and finish data acquisition modules ADAM.

In the project Data acquisition system of boiler №1 CHP (RU) to hands we get panel PC PPC-L126 and industrial PC on chassis IPC-6608 with processor board PCA-6753.

The industrial PC "PCA-6753" has next specification:

Central processor:	Low-consumption NS GXm-200 MHz (fanless)
Operational memory:	64 МБ DIMM SDRAM
Permanent memory:	SSD DiskOnChip® 2000, IDE (40 pin, UDMA 33, 256МБ IDE Flash)
Video subsystem:	CX5530 VGA/LCD and 18-bit LCD TFT
Audio subsystem:	no
Interfaces/ports:	10/100Mbps Ethernet (RTL-8139); RS-232 + RS-232/422/485 serial ports; 2 x USB 1.1 (host), IR port; LPT; Floppy
Power supply:	+5VDC ±10% @ 1.54A
Operating conditions:	0 ... 60°C

The controller is some old and has a low-performance for modern programs but for the tasks of execution environments of PLC it is suitable enough and on it was successfully downloaded firmware based on ALTLinux 5.1 packages base. At that about a real-time no a speech was by the high-precision timer (HPET) lack.

The panel PC "PPC-L126" has next specification:

Central processor:	VIA Eden 667 MHz
Operational memory:	Two 168-pin DIMM sockets (128 MB)
Permanent memory:	CompactFlash type I/II, IDE (44 pin, UDMA 33/66/100, 10 GB)
Video subsystem:	VIA Savage4 2D/3D/Video, sensor display 12.1", frontal panel: IP65/NEMA4
Audio subsystem:	AC97 Ver. 2.0
Interfaces/ports:	10/100Mbps Ethernet (RTL-8139); 3 x RS-232 + RS-232/422/485 serial ports; 2 x USB 1.1 (host); PS/2 keyboard and mouse; LPT
Power supply:	19VDC @ 3.3A
Operating conditions:	0 ... 40°C

The panel PC also has small resources for start modern environment then for installing on it was using the distribution "ALTLinux T6" with the actions to optimization:

• storage drive performance — for the data storage used the IDE hard-drive which ensure 10 MB/s, more then allowed CompactFlash;
• all spare services disable;
• the desktop environment TDE usage as lesser demanded to resources but more advanced; into the reserve has IceWM;
• building OpenSCADA without libraries Phonon and WebKit which too demanded to resources.

These actions allow fit the system to 128 MB of operational memory and get at that an advanced and productivity graphical environment. For the sensor display ELO configuration had using the driver "elographics" — "xorg-drv-elographics". To the configuration file "xorg.conf" added section:
Section "InputDevice" Identifier "ELO touchscreen" Driver "elographics" Option "Device" "/dev/ttyS3" Option "MinX" "300" Option "MaxX" "3800" Option "MinY" "3800" Option "MaxY" "300" Option "UntouchDelay" "10" Option "ReportDelay" "10" Option "SendCoreEvents" "yes" Option "ScreenNumber" "0" Option "ButtonNumber" "1" EndSection