Mittwoch, 13. Mai 2015

C/C++: data alignment in structs

What does alignment of data in structs mean? Consider the following struct:
 struct myStruct {  
      char aChar;  
      double aDouble;  
      uint32_t a32BitUInt;  
      bool aBool;  
 };  
What would you think the size of that struct is?

The C standard forbids the reordering of this struct like mentioned here and here.

Alignment means that the address of a variable starts at a memory location which is divisible by the size of the variable. So if you take a32BitUInt for example which is of type uint32_t this variable can be located at addresses which are equal to n * 4 because the size of the variable is 4 bytes.

Now what will happen in our case? The compiler will look at the variables in the struct and determine the largest size used which will be aDouble in this case. The compiler will put the struct at a memory location which is aligned to the size of the largest variable (address = n * 8, because sizeof(myStruct.aDouble) = 8).

The compiler isn't allowed to change the order of the variables in memory so aChar will have the lowest address. Lets assume the address for aChar was 0. The next possible address would be 1 but this is not aligned for double numbers so aDouble will be stored at address 8 because this address is divisable by sizeof(myStruct.aDouble) = 8.

a32BitUInt has a size of 4 bytes and the next possible address is 16 so this is an aligned address. 20 would be the next possible address and this is also aligned for bool which has a size of 1.

The size of the struct has to be a multiple of the largest variable size. So taking our example the current size would be 21 (addresses 0 .. 20). 21 (current size of struct) divided by 8 (size of aDouble) = 2,625. This would result in the compiler to add some padding so that the size would be ceil(2,625) * 8 bytes = 3 * 8 bytes = 24 bytes.

So here it is what it would look like in memory:

If printing out the values:
      myStruct myStructVar;  
      cout << "Size of char: " << sizeof(myStructVar.aChar) << endl;  
      cout << "Size of double: " << sizeof(myStructVar.aDouble) << endl;  
      cout << "Size of uint32_t: " << sizeof(myStructVar.a32BitUInt) << endl;  
      cout << "Size of bool: " << sizeof(myStructVar.aBool) << endl;  
      cout << "Size of struct: " << sizeof(myStructVar) << endl;  
      cout << "Start address of struct: " << &myStructVar << endl;  
it would look like this:
 Size of char: 1  
 Size of double: 8  
 Size of uint32_t: 4  
 Size of bool: 1  
 Size of struct: 24  
 Start address of struct: 0x7ffd147cedb0  

Dienstag, 12. Mai 2015

Qt: Setting dbus-properties and calling dbus-mehtods using QtDBus

The following examples on this page will set the "Powered" property for bluez to true for the first bluetooth adapter (hci0) found.

All examples use the following defines:
 #define BLUEZ_DBUS_SERVICE "org.bluez"  
 #define BLUEZ_DBUS_PATH "/org/bluez/hci0"  
 #define BLUEZ_DBUS_IF "org.bluez.Adapter1"  


Set a dbus property using QtDBus

   QDBusInterface ifc(  
         BLUEZ_DBUS_SERVICE,  
         BLUEZ_DBUS_PATH,  
         BLUEZ_DBUS_IF,  
         QDBusConnection::systemBus());  
   if ( ifc.isValid() ) {  
     ifc.setProperty("Powered", true);  
   }  

Calling methods with QtDBus

Example 1:
   QDBusConnection bus = QDBusConnection::systemBus();  
   if (!bus.isConnected())  
   {  
     qFatal("Cannot connect to the D-Bus session bus.");  
     return;  
   }  
   QDBusInterface dbus_iface(BLUEZ_DBUS_SERVICE,  
                BLUEZ_DBUS_PATH,  
                "org.freedesktop.DBus.Properties",  
                bus);  
   if ( dbus_iface.isValid() ) {  
     QDBusPendingReply<QVariantMap> reply = dbus_iface.asyncCall("Set",  
                                   BLUEZ_DBUS_IF,  
                                   "Powered",  
                                   QVariant::fromValue(QDBusVariant(true)));  
     reply.waitForFinished();  
   }  

Example 2:
   QDBusConnection bus = QDBusConnection::systemBus();  
   if (!bus.isConnected())  
   {  
     qFatal("Cannot connect to the D-Bus session bus.");  
     return;  
   }  
   QDBusMessage message = QDBusMessage::createMethodCall(BLUEZ_DBUS_SERVICE,  
                              BLUEZ_DBUS_PATH,  
                              "org.freedesktop.DBus.Properties",  
                              "Set");  
   QList<QVariant> arguments;  
   arguments << BLUEZ_DBUS_IF
        << "Powered"
        << QVariant::fromValue(QDBusVariant(true));  
   message.setArguments(arguments);  
   QDBusPendingReply<QVariantMap> reply = bus.asyncCall(message);  
   reply.waitForFinished();  

In case there is something not working the reply message can provide some usefull hints:
    qDebug() << reply.error().message();   

qdbusviewer

A really nice tool is the qdbusviewer which you can use to browse the dbus hierarchy:
For methods this tool will also show you types of the parameters.

Footnote

If the rfkill block bits are set for  the bluetooth devices you still have to unblock the bluetooth devices first:
$ rfkill unblock bluetooth  
At least on my system this has the side effect that the Powered property will be set to true. With the code above you can set the property to false without affecting the rfkill setting.

Donnerstag, 7. Mai 2015

Qt: Poweroff System

I'm currently working on an embedded project and wanted to have an option to poweroff the system with a keyboard shortcut. The system is using systemd as init-system.

The following two functions show how you can call the poweroff function:
 void MainWindow::poweroff()  
 {  
   QDBusConnection system = QDBusConnection::systemBus();  
   if (!system.isConnected())  
   {  
     qFatal("Cannot connect to the D-Bus session bus.");  
     return;  
   }  
   QDBusMessage message = QDBusMessage::createMethodCall("org.freedesktop.systemd1",  
                          "/org/freedesktop/systemd1",  
                          "org.freedesktop.systemd1.Manager",  
                          "PowerOff");  
   QDBusConnection::systemBus().asyncCall(message);  
 }  
 void MainWindow::poweroff_2()  
 {  
   QProcess *process = new QProcess(this);  
   QString program = "systemctl";  
   QStringList arguments("poweroff");  
   process->start(program, arguments);  
 }  
The first function will send a dbus message to systemd to poweroff the system while the second function will execute the systemctl command with poweroff as argument.

$ gdbus introspect --system --dest org.freedesktop.systemd1 --object-path /org/freedesktop/systemd1 will give you a nice overview of the available interface and methods available for systemd.

To assign a shortcut you can use the following functions (I'm calling these two functions in the constructor of my MainWindow class):
   new QShortcut(QKeySequence(Qt::CTRL + Qt::SHIFT + Qt::Key_F4), this, SLOT(poweroff()));  
   new QShortcut(QKeySequence(Qt::ALT + Qt::Key_F4), this, SLOT(poweroff_2()));  

Freitag, 24. April 2015

Yocto: INHERIT, inherit, include, require

For me it was a little bit confusing which directive I have to include in which file to include other files and which includes are valid. This is a little table I've made which shows the valid combinations. I give no promise though that this table is correct or complete but so far it worked for me:
Target: .bb .bbclass .conf .inc
.bb
.bbclass inherit inherit INHERIT
.conf
.inc
include
require
include
require
include
require
include
require

So for example if you are working on a .bb file you can use the inherit statement to include a .bbclass file.

inherit, INHERIT and include won't produce an error if the file does not exist, only the require statement will cause the build to file if the file does not exist!

More informations about the statements can be found on the following website: Locating Include and Class Files

Donnerstag, 23. April 2015

Yocto: Switching to Systemd

If you want to use Systemd to replace the default init-system you can create your own distribution file like this:
 require conf/distro/poky.conf  
 DISTRO = "systemd-distribution"  
 DISTRO_NAME = "Based on Poky which uses Systemd as init system "  
 DISTRO_VERSION = "1.0"  
 DISTRO_FEATURES_append = " systemd"  
 # uncomment the following two line two use directfb  
 #DISTRO_FEATURES_append = " directfb"  
 #DISTRO_FEATURES_remove = " x11 wayland"  
 DISTRO_EXTRA_RDEPENDS += "\  
   initscripts \  
   systemd \  
   systemd-analyze \  
   "   
 # Switch init-system to systemd  
 VIRTUAL-RUNTIME_init_manager = "systemd"  
 DISTRO_FEATURES_BACKFILL_CONSIDERED = "sysvinit"  
 VIRTUAL-RUNTIME_initscripts = ""  

Then you need to change the distribution name in your local.conf:
 DISTRO ?= "systemd-distribution"  

If you build your system it should now use Systemd as init-service.

This is the original site I've found this information: Yocto part I – baseline boot time

Donnerstag, 16. April 2015

VMware: Switching to paravirtual SCSI adapter for a CentOS vmware virtual machine

If you want to switch to the paravirtual SCSI adapter in your virtual machine because of performance reasons you can do it like this (this guide assumes you have configured your virtual hard drives as SCSI devices):
The first step is to make sure the paravirtual SCSI driver is included in your initial ramdisk. For this you have to edit /etc/dracut.conf and add the following line:
add_drivers+=" vmw_pvscsi "
and then you have to regenerate your initial ramdisk:
# mv /boot/initramfs-$(uname -r).img /boot/initramfs-$(uname -r).img.bak
# dracut -f
After you have shutdown your virtual machine you have to change the virtual SCSI adapter like this in your .vmx file:
scsi0.virtualDev = "pvscsi"
This has to be done for every adapter which should be converted to a paravirtual adapter.

This was tested on CentoOS 6 and 7.

Dienstag, 17. März 2015

Creating A Windows 7 SP1 USB Installer With Language Packs And Updates Integrated

Now that Windows 8.1 has been released for a while and Windows 10 is coming in the near future this information is probably a little dated but if you need to create a Windows 7 installer USB stick then hopefully this information will help you. You can only use the part you're interested, for example you might only be interested in integrating the updates and creating an installer ISO then you can simply skip the other parts. This guide is splitted into the following sections:
  • Needed tools
  • Integrating the language packs
  • Integrating the updates and creating the ISO
  • Creating a GPT formated USB stick

Needed Tools

  • Windows 7 ISO with SP1 integrated: You can get the English ISO here: Windows 7 SP1 ISO
  • 7-ZIP: Tool to extract files from downloaded ISO
  • Language pack you want to install:  Language packs (make sure you download the correct version: X86, X64)
  • exe2cab: Tool needed to convert language pack executables to cab files
  • DISM: I think this tool is included in Windows starting with Windows 7. If you don't have this tool installed you can try GetWaikTools to download it. This tool is needed to integrate the language packs.
  •  There are two options you can use to download the Windows Updates: You can either use a tool which uses updates list or use a tool which uses Windows Updates to check for the available updates. Each option does have it's own strength:
    • Tools which use updates list don't need a running version of your target Windows. Examples are:
    • Tools which use Windows Updates have the advantage that they use the same mechanism which Windows uses to determine the needed updates which means they will always list all updates which are currently available. The disadvantage is that you need to have a running version of your target Windows installation:
      • Portable Update: This tool will download the Windows updates as .cab files which seems to be faster to integrate then the .msu files downloaded by Windows Updates Downloader
  • optional: VMware Player: If you want to use Portable Update you can use this to install Windows inside a virtual machine to download the needed updates
  • NTLite: Integrates updates and creates ISO
  • Rufus: This tool will be used to create the bootable USB stick

Folder Structure

The folder structure should look like this:
 Explanation
  • IE11x64: If you want to integrate Internet Explorer 11 into your image the needed files will be put into this folder
  • Langpacks: Folder where you put your converted language packs .cab files
  • Langpacks_exe: Folder where you put your downloaded language packs .exe files
  • Updates: Folder where you put your update files
  • win7_pro_sp1_x64_en: Folder where you've unpacked your Windows 7 SP ISO
  • exe2cab.cmd: Small script which converts language pack exe files to cab files
  • exe2cab.exe: Tool to convert language pack exe files to cab files
  • langadd64-w7.cmd: Script which integrates language packs into your installer
  • win7_pro_sp1_x64_en.iso: The Windows 7 SP1 you've downloaded

Extract Windows ISO

Use 7-ZIP to extract the Windows ISO file into a subfolder (e.g. win7_pro_sp1_x64_en). If you want to be able to select the version (Starter, Home Basic, Home Premium, Professional, Ultimate) you have to delete the ei.cfg file which is located in the sources folder.

Integrating Language Packs

In the first step you have to download all the language packs you want to integrate into your Windows 7 installation. Put the downloaded language pack executables into the Langpacks_exe folder and then execute the following script (exe2cab.cmd) to convert the .exe to .cab files (unfortunately exe2cab.exe doesn't have a silent switch so you have to confirm each converted package:
 @echo off & setlocal enableextensions enabledelayedexpansion  
 set BASEDIR=%~dp0  
 set LANGPACKS=%BASEDIR%Langpacks  
 set LANGPACKS_EXE=%BASEDIR%Langpacks_exe  
 FOR /R %LANGPACKS_EXE% %%i IN (*.exe) DO (   
  %~dp0\exe2cab.exe %%i  
 )  
 move /Y %LANGPACKS_EXE%\*.cab %LANGPACKS%  

Caveat!: If you want to integrate a language which needs additional font packages then please read the note on the following page: Add a Language to Windows Setup

Create the following langadd64-w7.cmd in your base directory (make sure to adjust the SRC path to the path where you've extracted the Windows 7 SP1 installation files):
 @echo off & setlocal enableextensions enabledelayedexpansion  
 set BASEDIR=%~dp0  
 rem set paths to windows sources and language packs  
 set SRC=%BASEDIR%win7_pro_sp1_x64_en  
 set LANGPACKS=%BASEDIR%Langpacks  
 set MOUNT=%BASEDIR%Mount  
 set EXT=%BASEDIR%Extract  
 if exist "%MOUNT%" rd /s /q "%MOUNT%"  
 mkdir "%MOUNT%"
 echo "Integrate language packs to all editions"
 for /l %%e in (1, 1, 4) do (  
  echo "Mounting install.wim(%%e)"  
  "dism.exe" /mount-wim /wimfile:"%SRC%\sources\install.wim" /index:%%e /mountdir:"%MOUNT%"  
  FOR /F "usebackq" %%i IN (`dir /b "%LANGPACKS%"`) DO (   
   "dism.exe" /Image:"%MOUNT%" /Add-Package /PackagePath:"%LANGPACKS%\%%i"  
   "dism.exe" /Image:"%MOUNT%" /Gen-LangIni /Distribution:"%SRC%"  
  )  
  "dism.exe" /Image:"%MOUNT%" /Get-Packages /Format:Table  
  "dism.exe" /Image:"%MOUNT%" /Get-Intl  
  echo "Unmounting install.wim(%%e)"  
  "dism.exe" /unmount-wim /mountdir:"%MOUNT%" /commit  
 )  
 echo "Mounting boot.wim"  
 "dism.exe" /mount-wim /wimfile:"%SRC%\sources\boot.wim" /index:2 /mountdir:"%MOUNT%"  
 FOR /F "usebackq" %%i IN (`dir /b "%LANGPACKS%"`) DO (   
  if exist "%EXT%" rd /s /q "%EXT%"  
  mkdir "%EXT%"  
  echo "Extract %%i language pack"  
  expand "%LANGPACKS%\%%i" -f:* "%EXT%" > NUL  
  call :getlocale  
  echo "Copy language pack files to boot.wim and windows sources"  
  xcopy "%EXT%\setup\sources\*" "%SRC%\sources\" /cherkyi > NUL  
  xcopy "%EXT%\sources\license\*" "%SRC%\sources\license\" /cherkyi > NUL  
  xcopy "%EXT%\setup\sources\*" "%MOUNT%\sources" /cherkyi > NUL  
  xcopy "%EXT%\sources\license\*" "%MOUNT%\sources" /cherkyi > NUL  
 )  
 copy "%SRC%\sources\lang.ini" "%MOUNT%\sources\lang.ini" > NUL  
 echo "Unmounting boot.wim"  
 "dism.exe" /unmount-wim /mountdir:"%MOUNT%" /commit  
 rd /s /q "%MOUNT%"  
 rd /s /q "%EXT%"  
 echo End of language pack integration  
 pause  
 exit /b
  
 :getlocale  
 set currarea=  
 for /f "usebackq delims=" %%a in ("%EXT%\langcfg.ini") do (  
   set ln=%%a  
   if "x!ln:~0,1!"=="x[" (  
     set currarea=!ln!  
   ) else (  
     for /f "tokens=1,2 delims==" %%b in ("!ln!") do (  
       set currkey=%%b  
       set currval=%%c  
       if "x[LangInfo]"=="x!currarea!" if "xLanguage"=="x!currkey!" (  
         set LANG=!currval!  
       )  
     )  
   )  
 )  
 exit /b  
Open a command prompt with administrative rights and cd into the directory where you've place the  langadd64-w7.cmd script and the simply run the script which will take a while to complete (make sure dism can be found in the path or place dism.exe into the same folder where you've put langadd64-w7.cmd).

Downloading Windows Updates

Here two options are described for downloading the Windows updates. The advantages and disadvantages of each options is described in the Needed Tools section.

Windows Updates Downloader

Download the Windows Updates Downloader and the updates file for your Windows version and select the updates which you want to integrate and download the updates.

Portable Update

When you want to use the portable udpate tool you first have to install Windows inside a virtual machine. You can either use the original Windows 7 ISO if you don't want to integrate any language packs or use NTlite to create an ISO file:
  • Open NTLite and select Add and select the folder where you've extracted your installation files
  • Select Create ISO to create a new ISO file
Install Windows 7 inside a virtual machine using either this image file or the original ISO depending on wether you want or do not want to integrate language packs.

After you have finished installing it into the virtual machine you can enable shared folders to easily move files between the virtual machine and the host. Make sure you select the option Map as a network driver in Windows guests as this will make it easier for you to transfer the files.

Now download, extract and run Portable Update inside your virtual machine and download the available updates. When you look at the downloaded files you can see that there are .cab and .exe files in your folder:
  • Put all the files into the Updates folder
  • When you go to the Updates folder you will see that some are additional language pack files. In File Explorer select to show File Description and then move all language pack files into the Langpacks_exe folder

 Integrate Internet Explorer 11

 Extract the Internet Explorer installation file like this:
 ie11-windows6.1-x64-en-us_ddec9ddc256ffa7d97831af148f6cc45130c6857.exe /x:<project root>\IE11x64  


Integrate Updates And Drivers

Unfortunately not all updates can be integrated using NTLite. You can try to convert the executables using the exe2cab tool which will work for a few selected packages and some of those can than be integrated into the installer.

 For the next steps you need NTLite:
  • Add the folder where you've extracted your Windows installation files
  • Load the edition you want to modify: In the source list you will see the editions listed in the install.wim. Select the edition you want to modify and Load the image.
  • Go to the Updates tab and add the Updates folder (this is there folder where you have copied your update files) and then the IE11x64 (this is the folder where you have extracted the Internet Explorer 11 installation files) folder. NTLite will show you a warning that some updates can not be integrated. This is ok because not all updates can be slipstreamed.
  • Go to the Drivers tab and add the drivers you want to include
  • Go to the Apply tab and start the integration process. You can select to either keep only the selected edition (Export image selected) or keep all editions (Export image deselected)
  • If you want to keep the other editions then you have to repeat these steps for the other editions also

Create ISO

Use NTLite to create the ISO file. You can either select the image in the Image history in the Source tab and then select Create ISO or you can select Create ISO image on the Apply tab when you have loaded a .wim file.

Creating GPT Formated USB Stick 

You can create a bootable USB very easlily using the Rufus tool. Plug an USB stick into your computer and start Rufus. Make sure to select the USB stick as Device. If you want to install Windows 7 on an UEFI computer with a GPT formatted or empty disk select the GPT partition scheme for UEFI computer. If you're booting using the old BIOS method or use a MBR formated disk chose MBR partition scheme for BIOS or UEFI computers.
Select the ISO file which was created by NTLite and start the process.

Additional Informations

Unfortunately this is far from perfect. On my test installation there were still 61 Windows updates available after a fresh install. This is because it's not possible to automatically integrate the .exe files updates (e.g. .NET update) and some of the updates simply can't be integrated for whatever reason.

I have gathered the informations here from various places:
  • Raymond: Instructions on how to integrate Windows updates into installation sources
  • nextofwindows: Instructions on how to create UEFI bootable USB stick
  • Vistalizer: Instructions on how to convert the language EXE files to CAB files; script for integrating language pack
  • Good to Know Database: Script which integrates language packs into install sources
  • Technet: Instructions on how to integrate language pack into installations sources
  • Slipstream Internet Explorer 11: Tutorial for integrating Internet Explorer 11


Mittwoch, 4. März 2015

Setting a static network address on the BeagleBone

After several unsuccessful attempts using connman and network-manager I've finally managed to boot the beagle board with a static IP address. First step was to remove wicd, connman and network-manager from the BeagleBone. I've tried to set the IP address using the nmcli tool to configure the network-manager but unfortunately the version installed on the BeagleBone didn't support the modification of connections. I then tried to create a configuration file for the usb0 ethernet but network-manager somehow ignored it.

When the BeagleBone boots systemd will execute the boot_scripts.service script which in turn will call the boot_scripts.sh script in /etc/init.d/.  I have no idea where the boot_scripts.service is located, but my guess is that systemd converts old InitV scripts it finds in /etc/rc*.d  to systemd services. But that would not explain why ifup\@.service, which will be shown later, gets executed. The boot_scripts.sh will then call either am335x_evm.sh or omap3_beagle.sh depending on the type of the board. On the BeagleBone Black this would be the am335x_evm.sh script.

Now to enable static IP configuration you have to comment out all lines in am335x_evm.sh (located in /opt/scripts/boot/) which will set the IP address and start the udhcp server like this:
 #sed -i -e 's:DHCPD_ENABLED="no":#DHCPD_ENABLED="no":g' /etc/default/udhcpd  
 ##Distro default...  
 #deb_udhcpd=$(cat /etc/udhcpd.conf | grep Sample || true)  
 #if [ "${deb_udhcpd}" ] ; then  
 #  mv /etc/udhcpd.conf /etc/udhcpd.conf.bak  
 #fi  
 #if [ ! -f /etc/udhcpd.conf ] ; then  
 #  echo "start   192.168.7.1" > /etc/udhcpd.conf  
 #  echo "end    192.168.7.1" >> /etc/udhcpd.conf  
 #  echo "interface usb0" >> /etc/udhcpd.conf  
 #  echo "max_leases 1" >> /etc/udhcpd.conf  
 #  echo "option subnet 255.255.255.252" >> /etc/udhcpd.conf  
 #fi  
 #/etc/init.d/udhcpd restart  
 #/sbin/ifconfig usb0 192.168.7.2 netmask 255.255.255.252  
 #/usr/sbin/udhcpd -S /etc/udhcpd.conf  
 #eth0_addr=$(ip addr list eth0 |grep "inet " |cut -d' ' -f6|cut -d/ -f1)  
 #usb0_addr=$(ip addr list usb0 |grep "inet " |cut -d' ' -f6|cut -d/ -f1)  
 #wlan0_addr=$(ip addr list wlan0 |grep "inet " |cut -d' ' -f6|cut -d/ -f1)  
 #sed -i -e '/Address/d' /etc/issue  
 i#f [ ! "x${eth0_addr}" = "x" ] ; then  
 #  echo "The IP Address for eth0 is: ${eth0_addr}" >> /etc/issue  
 #fi  
 #if [ ! "x${wlan0_addr}" = "x" ] ; then  
 #  echo "The IP Address for wlan0 is: ${wlan0_addr}" >> /etc/issue  
 #fi  
 #if [ ! "x${usb0_addr}" = "x" ] ; then  
 #  echo "The IP Address for usb0 is: ${usb0_addr}" >> /etc/issue  
 #fi  


Then create /etc/systemd/system/ifup\@.service with the following contents:
 [Unit]  
 Description=ifup for %I  
 After=local-fs.target  
 [Service]  
 ExecStart=/sbin/ifup --allow=auto %I  
 ExecStop=/sbin/ifdown %I  
 RemainAfterExit=true  
The strange thing is that you don't have to enable this service. Just create the file and it will be called for the interfaces automatically. If anyone knows why then please let me know!

Now you have to configure your interface in /etc/network/interfaces. Here is my example configuration for the usb0 network interface:
 auto usb0  
 allow-hotplug usb0  
 iface usb0 inet static  
   address 10.42.0.252  a
   netmask 255.255.255.0  
   network 10.42.0.0  
   gateway 10.42.0.1  
   dns-nameservers 10.42.0.1  
You need to have auto usb0 in your configuration because the service script will only start interfaces with this attribute.

Restart you BeagleBone and hopefully it works for you also.