I’m back!

I’m back! After 2 1/2 years! and about to get an i7 laptop too!  Pretty amazing, going from a P6100 with CPU mark of about 1350 to an i7 with a CPU mark of 7000! That’s almost 5X performance for about 50% extra $!!!

Cloud digression

Digression to running enterprise cloud (Eucalyptus based) using Ubuntu:

https://help.ubuntu.com/community/UEC/

http://fnords.wordpress.com/2009/10/04/run-your-own-uec-part-1/

Cloud over Ubuntu over Virtualbox over Windows 7?? We’ll see how slow this  will get on my new core i3 machine.

X Programming Ref

Xlib programming Reference:

PDF:     http://www.x.org/archive/X11R7.0/doc/PDF/xlib.pdf

HTML: http://tronche.com/gui/x/xlib/function-index.html

For indirect rendering through X which the softpipe uses, calls to Xlib are in drisw_glx.c module in  src/glx, with the indirection table created when __GLXInitialize is called to initialized GL over X.

Also, an architectural diagram of how VitualGL uses GLX to bind OpenGL rendering context to X windows.

http://en.wikipedia.org/wiki/VirtualGL

Found missing link

The missing link to this whole Gallium picture is found:  the source code to the kernel DRM device driver (not libdrm which is just its user space counterpart, not the stuff in mesa winsys drm directory, which are just redirections), the module that services ioctl’s, the module that calls moduleInit()… And nowhere was this piece talked about in mesa3d portal nor DRI wiki!

Hats off to the folks at Genode Labs, whose comments on Gallium3D was concise, precise and best articulated among all Gallium docs I’ve come across in the past 6 months.  Their comment on porting Gallium was right on:

“…Gallium3D interplays with the graphics execution manager (GEM) in the kernel. So porting Gallium3D to a new OS involves porting GEM as well. For the initial step, Intel GMA was selected as the first series of GPUs to support. Thanks to Genode’s flexible C runtime and the dynamic linker, the most challenging part of porting the Gallium3D libraries was understanding the relationship between the various parts of Mesa, but there were no technical difficulties. The GEM part was significantly more labor intensive because GEM normally relies on Linux kernel interfaces….”

http://www.osnews.com/story/23671/Genode_10_08_Comes_with_Gallium3D_MadWifi_Qt4_6_3

The most challenging part of working with Gallium3D, as the above message commented,  has been indeed finding out what pieces are involved in the overall picture, and not any technical difficulty.  Where is that document that should have been there to explain control flow between Gallium, libdrm and kernel DRM driver?

Gallium3D trace driver

See README file (does not exist in pre-7.9 tree) in 7.9 source tree trace driver directory for instructions. Basically:

1) set the name of the trace output file in GALLIUM_TRACE  environment var

2) call trace_screen_create() passing it the screen ptr created for the actual device

3) output file is in XML format and needs the formatting style sheet. Put trace.xsl in the same directory as outputfile and open output file with IE or Firefox.

Why isn’t this kind of information made more accessible from DRI wiki or mesa3d portal?

Trace driver background:

http://jrfonseca.blogspot.com/2008/09/tracing-gallium3d.html

more notes on building and running 7.9 and 7.8.2

Building driver framework for immerse:

1. Build mesa 7.9 gallium framework

export PKG_CONFIG_PATH=/usr/local/lib/pkgconfig:$PKG_CONFIG_PATH’
cd ~/work/mesa/7.9
./configure –enable-debug –enable-gallium-i915 –enable-gallium-swrast
make

2. Build immerse winsys (libimmsw.a)

This builds immerse winsys, baically buffer management functions. Currently builds libimmsw.a, based on winsys/i915/sw which allocs from system heap. When there is immerse hardware and hardware dependent buffer management, will have to switch to building ~/work/mesa/src/gallium/winsys/immerse/drm/libimmdrm.a

cd ~/work/mesa/src/gallium/winsys/immerse
make

3. Build immerse pipe library framework (libimm.a)

Stubs using i915 code.  To be replaced after mapping to immerse registers and descriptors is clarified.

cd ~/work/mesa/src/gallium/drivers/immerse
make

4. Build immerse gallium driver (immerse_dri.so)

cd ~/work/mesa/src/gallium/target/dri-immerse
make

Note:

The above process builds the Gallium softpipe as well.  To run MESA apps (including progs in test2) against it, set the following env vars:

export LD_LIBRARY_PATH=<mesa path>/lib
export LIBGL_DRIVERS_PATH=<mesa path>/lib (if using DRI drivers)
export LD_PRELOAD=<mesa path>/lib/libGL.so.1

Make sure the correct modules are used:

ldd <application>

The softpipe was used extensively in helping figure out how the pipe context rendering functions were called by running mesa apps and tracing execution (see sample traces in ~/traces).

Building softpipe driver in 7.8.2

It’s not straight forward in figuring how to build the softpipe driver in 7.8.2. You’d have to read the softpipe code to find out how it’s initialized and figure that it’s invoked from the xlib module in winsys, then read the autoconf generated configure script to figure how to build it. Anyways, here’s the intuitive command option – no wonder they had to change it in 7.9.

./configure –enable-debug –enable-gallium-swrast –with-driver=xlib

And it doesn’t generate a separate softpipe driver .so file like 7.9 but is all linked into libGL. So make sure to load the right one:

export LIBGL_DRIVERS_PATH=/home/parkson/work/mesa/Mesa-7.8.2/lib

export LD_LIBRARY_PATH=/home/parkson/work/mesa/Mesa-7.8.2/lib

export LD_PRELOAD=/home/parkson/work/mesa/Mesa-7.8.2/lib/gallium/libGL.so.1

VirtualBox, Ubuntu, and Mesa3D

Switched over to virtualbox because it starts and stops vm’s much faster than vmware and does not cause thrashing on my athlon 64×2 machine with 3Gb memory even after running for days.  The perk was that setting it up to auto-detect and use full resolution of the monitor on the host was pretty easy (at least with ubuntu – but a pain with Fedora).  So I’ve been running mesa on ubuntu in virtualbox. Much friendlier and productive environment.

Virtualbox setup (using 3.2.8):

http://www.virtualbox.org/wiki/Downloads

note: install GuestInstallations under VictualBox devices to enable auto detect host monitor resolution

Ubuntu desktop edition (using 10.04):

http://www.ubuntu.com/desktop/get-ubuntu/download

Meas3d (using 7.8.2)

ftp://ftp.freedesktop.org/pub/mesa/

note:  xorg development tools and g++ needs to be installed to build mesa and c-model.

sudo apt-get install xorg-dev

sudo apt-get install g++D

Don’t try installing xorg dev tools piecemeal – figuring dependencies will kill you

Network problem

Wifi network on this side of the house has been coming to a grinding halt intermittently for days. Thought it was power line issues affecting the WGX102 but problem persisted after adding Wifi universal repeater and access point. Now suspecting wifi channel interference but too many access points and too few channel id (1, 6, 11) available. Trying out new tool to see if there are interfering access points in the area.

http://www.nirsoft.net/utils/wireless_network_view.html

more on gallium driver initialization

loading and initializing Gallium driver

1) program calls OpenGL API

main cube.c:90                 glutCreateWindow glut_win.c:731

__glutCreateWindow      glut_win.c:522

__glutDetermineVisual  glut_win.c:401

__glutGetVisualInfo        glut_win.c:383

getVisualInfoRGB             glut_win.c:348

2) OpenGL API calls GLX extension which loads Gallium Driver and binds OpenGL, DRI state checker, and HW pipe driver.

__glXIintialize() glxext.c

– init DRI per display data & functions:

– Create display structs by calling:

– dri2CreateDisplay glx/dri2_glx.c

– driCreateDisplay glx/dri_glx.c

– drwswCreateDisplay glx/drisw_glx.c

& save in display struct ptr to screen create functions

– dri2Display->createScreen = dri2CreateScreen (dir2_glx.c)

– driswDisplay->createScreen = driCreateScreen (dirsw_glx.c)

– then calls AllocAndFetchScreenConfigs to create screen, calling:

– dri2Display->createScreen()   (for hardware)

– driswDisplay->createScreen() (for softpipe)

the screen screate functions:

– dlloads dri/drisw driver module and open driver

– dlsym DRI ext. table DRI_DRIVER_EXTENSIONS (__driDriverExtenaions)

– calls createNewScreen() in DRI driver extension table (in DRI st tracker)

– dri2CreateNewScreen (hw pipes – dri_util.c)

– driCreateNewScreen (soft pipe – dirsw_util.c)

these saves ptr to DRI Driver Vitrual Function table (in DRI st tracker)

then calls the init function in the table,

– driDriverAPI.InitScreen2 (dri2_init_screen – dri2.c for hw pipes)

– driDriverAPI.InitScreen (drisw_init_screen – drisw.c  for soft pipe)

which binds DRI st tracker and OpenGL, and

calls create_screen ptr in Driver Descriptor Table (in individual driver)

————————————————————–

DRI Driver Extension Tables

~/src/gallium/state_trackers/dri/drm/dri2.c (for hw pipe drivers)

/* This is the table of extensions that the loader will dlsym() for. */

PUBLIC const __DRIextension *__driDriverExtensions[] = {

&driCoreExtension.base,

&driLegacyExtension.base,

&driDRI2Extension.base,

NULL

};

~/src/gallium/state_trackers/dri/sw/drisw.c (for softpipe)

/* This is the table of extensions that the loader will dlsym() for. */

PUBLIC const __DRIextension *__driDriverExtensions[] = {

&driCoreExtension.base,

&driSWRastExtension.base,

NULL

};

————————————————————–

* note: The DRI interface table is defined in dri_util.c for gallium DRI drivers

The ones defined in ~/src/mesa/drivers/dri/xxx are for pre-gallium DRI drivers

~/src/mesa/drivers/dri/common/dri_util.c

/** DRI2 interface */

const __DRIdri2Extension driDRI2Extension = {

{ __DRI_DRI2, __DRI_DRI2_VERSION },

dri2CreateNewScreen, /* called by dri2CreateScreen */

dri2CreateNewDrawable,

dri2CreateNewContext,

dri2GetAPIMask,

dri2CreateNewContextForAPI

};

~/src/mesa/drivers/dri/common/drisw_util.c

const __DRIswrastExtension driSWRastExtension = {

{ __DRI_SWRAST, __DRI_SWRAST_VERSION },

driCreateNewScreen, /* called by driCreateScreen */

driCreateNewDrawable

};

————————————————————–

DRI Driver Virtual function tables

~/src/gallium/state_trackers/dri/drm/dri2.c (for hardware pipes)

/* DRI driver virtual function table. */

const struct __DriverAPIRec driDriverAPI = {

.InitScreen = NULL,

.InitScreen2 = dri2_init_screen,

.DestroyScreen = dri_destroy_screen,

.CreateContext = dri2_create_context,

.DestroyContext = dri_destroy_context,

.CreateBuffer = dri2_create_buffer,

.DestroyBuffer = dri_destroy_buffer,

.MakeCurrent = dri_make_current,

.UnbindContext = dri_unbind_context,

.GetSwapInfo = NULL,

.GetDrawableMSC = NULL,

.WaitForMSC = NULL,

.SwapBuffers = NULL,

.CopySubBuffer = NULL,

};

~/src/gallium/state_trackers/dri/sw/drisw.c (for softpipe)

const struct __DriverAPIRec driDriverAPI = {

.InitScreen = drisw_init_screen,

.DestroyScreen = dri_destroy_screen,

.CreateContext = dri_create_context,

.DestroyContext = dri_destroy_context,

.CreateBuffer = drisw_create_buffer,

.DestroyBuffer = dri_destroy_buffer,

.MakeCurrent = dri_make_current,

.UnbindContext = dri_unbind_context,

.SwapBuffers = drisw_swap_buffers,

};

————————————————————–

Gallium Driver Descriptor Table (used for HW pipes only)

e.g. gallium/targets/dri-i915/target.c

defines:

create_screen()                                         init i915 winsys and screen

i915_drm_winsys_create()            winsys/i915/drm/i915_drm_winsys.c

i915_screen_create()                       drivers/i915/i915_screen.c:

driver_descriptor global struc:

driver_descriptor.name = i915

.driver_name = i915;

.create-screen = create screen

————————————————————–

Context creation flows through in similar manner:

dri2_glx.c:

static const struct glx_screen_vtable dri2_screen_vtable = {

dri2_create_context

};

glxcmds.c: glXCreateContext

-> glxcmds: CreateContext

-> dri2.c: dri2_create_context (via dri2_screen_vtable)

-> …

The actual hw rendering interface hooks into the context jump table setup here.

More Mesa/Gallium3D background info

Graphics pipeline:

Graphics pipeline – Wikipedia, the free encyclopedia

graphics pipeline- Information from Answers.com

Shader – Wikipedia, the free encyclopedia

MIT OpenCourseWare – Electrical Engineering and Computer Science – 6.837 Computer Graphics, Fall 2003 – Home

DRM/DRI:

How DRI and DRM Work – WizBlog

Introduction to the Direct Rendering Infrastructure

The Direct Rendering Manager- Kernel Support for the DRI

Direct Rendering Infrastructure – Wikipedia, the free encyclopedia

DRI Wiki – libGL

DRI2 Protocol Spec Draft v2 released [LWN.net]

DRI2 Protocol Specification

Dri Troubleshooting DRI Profiling

EGL:

Mesa EGL

freedesktop.org – Software/gallium/EGLReferenceDrivers

Gallium and OpenGL:

GALLIUM SOURCE INFO Gallium3D- Main Page

OpenGL

GLX

Gallium3D – Wikipedia, the free encyclopedia

Mesa Home Page (traditional make, scon, autoconf)

DRI Driver Installation:

Intel Linux Graphics: Installation

Howto Install ATI Drivers (Mesa 3D DRI Experimental) on Fedora 12

nouveau Wiki – GalliumHowto