First Flight

From Infinity Wiki
Revision as of 05:52, 23 June 2016 by GaryBenson (Talk | contribs) (Issues)

Jump to: navigation, search

First Flight is the first full-system preview release of Infinity. First Flight is just enough for GDB to attach to a live process or start with a core file. info threads should work, print errno will not. Infinity First Flight was released June 13, 2016.

Building Infinity First Flight

There are two things you need to build to try Infinity First Flight: a glibc with Infinity notes, and the shim libthread_db that accesses them. You will also need something that uses libthread_db, for example GDB.

Create a test build of glibc with Infinity notes

To build glibc with Infinity notes you first need to install I8C. For First Flight you need I8C 0.0.3, which you can install with PIP:

pip install --user i8c==0.0.3

or you can build from source:

git clone https://gitlab.com/gbenson/i8c.git
cd i8c
git branch i8c-0.0.3 i8c-0.0.3-release
git checkout i8c-0.0.3
python setup.py test  # optional
python setup.py install --user

The command i8c needs to be in your path before you continue. Either of the above sequences will install I8C in ~/.local/bin/i8c, so you may need to add ~/.local/bin to your path. Alternatively you can omit the --user and install I8C globally on your system. However you install it, you need to be able to enter this exact command:

i8c --version | head -n 1

and see this exact output:

I8C 0.0.3

Once you have I8C installed you can proceed to build glibc:

mkdir -p glibc/build
git clone https://gitlab.com/gbenson/glibc.git glibc/src
cd glibc/src
git branch first-flight infinity-first-flight
git checkout first-flight
cd ../build
../src/configure --prefix=/usr --with-infinity
make

--with-infinity is assumed if you have I8C installed, but specifiying it explicitly here forces configure to check the version for you.

You should be able to see Infinity notes in libpthread.so if everything worked:

readelf -n nptl/libpthread.so

should list several notes with owner "GNU" and unknown type 5:

Displaying notes found at file offset 0x0002144c with length 0x00000378:
  Owner                 Data size	Description
  GNU                  0x0000006d	Unknown note type: (0x00000005)
  GNU                  0x0000007d	Unknown note type: (0x00000005)
  GNU                  0x000000cc	Unknown note type: (0x00000005)
  GNU                  0x0000009f	Unknown note type: (0x00000005)
  GNU                  0x0000005f	Unknown note type: (0x00000005)
  GNU                  0x0000005b	Unknown note type: (0x00000005)

See https://sourceware.org/glibc/wiki/Testing/Builds for more information about glibc test builds.

Build libi8x with the shim libthread_db

To build libi8x with the shim libthread_db you first need to install the elfutils libelf development files. For Fedora and RHEL this is the elfutils-libelf-devel RPM. For Debian and Ubuntu the package you need is libelf-dev. Either way, once done you should see /usr/include/libelf.h on your system.

Once that's there, clone and build libi8x:

 git clone https://gitlab.com/gbenson/libi8x.git
 cd libi8x
 git branch libi8x-0.0.2 libi8x-0.0.2-release
 git checkout libi8x-0.0.2
 ./autogen.sh
 ./configure
 make

You should see an examples/libthread_db.so.1 symbolic link if it worked.

Things to do with Infinity First Flight

Run the note tester on the glibc you built

The note tester I8X is hooked into glibc's make check, but lets run it directly:

 cd /path/to/glibc/build
 i8x -I. -i nptl/libpthread.so ../src/nptl/tst-infinity-*.py

You should see some output as to the result of the tests. All tests should pass, and there should be no coverage errors reported.

To run with glibc's make check, try this:

  cd /path/to/glibc/build
  make subdirs=nptl check

I8X's output will be in nptl/tst-infinity.out, and the result that glibc's make check consumes will be in nptl/tst-infinity.test-result.

Attach to a live process with GDB

Start a process using the glibc with Infinity notes. I'm going to use Python:

/path/to/glibc/build/testrun.sh /usr/bin/python

Enter the following into Python to make it print its process ID:

 import os
 os.getpid()

In another window, start GDB.

 gdb

Make sure you're using GDB 7.9 or newer; older versions require libthread_db functionality the shim doesn't implement.

Assuming your GDB is new enough, tell it to use the shim libthread_db:

 set auto-load safe-path /path/to/libi8x/examples
 set libthread-db-search-path /path/to/libi8x/examples

Now attach to the Python process you created:

 attach PID

You should see a "Welcome to Infinity" message. Try listing threads:

 info threads

There is probably just the one. Type continue into GDB, then switch back to the Python window and start another thread:

 import threading
 threading.Timer(60, dir).start()

You should see a "new thread" message in the GDB window. Press Ctrl-C in the GDB window then try info threads again. There should now be two threads.

Debug a core file with GDB

Start a process using the glibc with Infinity notes. Again, I'm going to use Python:

/path/to/glibc/build/testrun.sh /usr/bin/python

Make Python print its PID, then start a couple of threads:

 import os
 os.getpid()
 import threading
 threading.Timer(60, dir).start()  # as many times as you like

In another window, use GDB to generate a core file of that process and then kill it:

 gdb -batch -p PID -ex gcore -ex kill

The Python process should now be gone, and you should find a core file wherever you ran it from. Now start GDB:

 gdb

Tell GDB to use the shim libthread_db, and load the executable and core file:

 set auto-load safe-path /path/to/libi8x/examples
 set libthread-db-search-path /path/to/libi8x/examples
 file /usr/bin/python
 core core.PID

You should see a "Welcome to Infinity" message as before.

Trace note bytecode as it executes

Try the #Attach to a live process with GDB or #Debug a core file with GDB example, but start GDB like this:

I8X_DEBUG=1 I8X_LOG=8 gdb

This time you should see a huge amount of debug information. Before the "Welcome to Infinity" message you'll see the various note functions being loaded and rewritten. libpthread::thr_get_lwpid(p)ii looks like this on my machine, for example:

libi8x: i8x_code_dump_itable:   0x21701: DW_OP_dup                => 0x21702
libi8x: i8x_code_dump_itable:   0x21702: DW_OP_bra 9              => 0x21705, 0x2170e
libi8x: i8x_code_dump_itable:   0x21705: I8_OP_load_external 1    => 0x21708
libi8x: i8x_code_dump_itable:   0x21708: I8_OP_call               => 0x2170a
libi8x: i8x_code_dump_itable:   0x2170a: DW_OP_lit0               => 0x2170b
libi8x: i8x_code_dump_itable:   0x2170b: DW_OP_skip 7             => 0x21715
libi8x: i8x_code_dump_itable:   0x2170e: DW_OP_plus_uconst 208    => 0x21711
libi8x: i8x_code_dump_itable:   0x21711: I8_OP_deref_int -32      => 0x21714
libi8x: i8x_code_dump_itable:   0x21714: DW_OP_lit0               => 0x21715
libi8x: i8x_code_dump_itable:   0x21715: I8X_OP_return

The hexadecimal numbers are offsets into the note provider, in this case libpthread.so. You can see the operations and any operands, and the final numbers (after the "=>") are the offset of the next instruction or instructions to be executed.

After the "Welcome to Infinity" message you'll see traces of the various functions as they execute:

libi8x: i8x_xctx_trace: libpthread::thr_get_lwpid(p)ii             0x21701	DW_OP_dup                [1]	0x4eaa68c0        	----------        
libi8x: i8x_xctx_trace: libpthread::thr_get_lwpid(p)ii             0x21702	DW_OP_bra                [2]	0x4eaa68c0        	0x4eaa68c0        
libi8x: i8x_xctx_trace: libpthread::thr_get_lwpid(p)ii             0x2170e	DW_OP_plus_uconst        [1]	0x4eaa68c0        	----------        
libi8x: i8x_xctx_trace: libpthread::thr_get_lwpid(p)ii             0x21711	I8X_OP_deref_i32n        [1]	0x4eaa6990        	----------        
libi8x: i8x_xctx_trace: libpthread::thr_get_lwpid(p)ii             0x21714	DW_OP_lit0               [1]	0x00003a2c        	----------        
libi8x: i8x_xctx_trace: libpthread::thr_get_lwpid(p)ii             0x21715	I8X_OP_return            [2]	0x00000000        	0x00003a2c

You can see the offsets and operations as before. The numbers in square brackets are the stack depth before the operation completed, and the final two columns are the top two entries in the stack, again before the operation completed.

Debug a static executable with GDB

In libi8x's "contrib" directory is a simple multithreaded example program and a script to build it. Run:

contrib/build-threads-example.sh /path/to/glibc/build

This should build contrib/threads-static amongst other things. Use readelf to check it has notes:

readelf -n contrib/threads-static

As with glibc's libpthread.so, this should list several notes with owner "GNU" and unknown type 5:

Displaying notes found at file offset 0x000c64fc with length 0x00000378:
  Owner                 Data size	Description
  GNU                  0x0000006d	Unknown note type: (0x00000005)
  GNU                  0x0000007d	Unknown note type: (0x00000005)
  GNU                  0x000000cc	Unknown note type: (0x00000005)
  GNU                  0x0000009f	Unknown note type: (0x00000005)
  GNU                  0x0000005f	Unknown note type: (0x00000005)
  GNU                  0x0000005b	Unknown note type: (0x00000005)

Now run that executable:

contrib/threads-static

It should print its process ID. Start GDB and attach to it:

set auto-load safe-path /path/to/libi8x/examples
set libthread-db-search-path /path/to/libi8x/examples
attach PID

You should see the "Welcome to Infinity" message, and info threads should list two threads.

Issues

  • First Flight doesn't work when the process being debugged has a different word size to libi8x. You cannot attach to an i686 process with an x86_64 GDB, for example.
  • First Flight libi8x doesn't build on 32-bit platforms.
  • With core files, First Flight requires executables and shared libraries to be in the locations they were when the executable ran. You can't debug with the files in a sysroot in GDB, for example.