Debug PlayStation3

Since I did a similar post for the Xbox 360 XDK a while back and since this is intended to be mostly a developer-oriented blog, I thought it would be in keeping with that spirit to let people have a general idea of what to expect from a debug PS3 – which recently arrived.

Photo album: All of the screenshots linked to below (and more) can be viewed separately in this photo album here.

Profiling/debugging

There are some limitations as to what you can do with regards to profiling – even on a Debugging Station. Code instrumentation through gcov is entirely possible, but this was not very appealing to me anymore given that we have been able to do this more or less even on Retail with Themaister’s net-stdio implementation. Gprof seems to have been made unavailable as Sony started moving more and more away from the open source GNU toolchain they originally based their development environment on. What’s left is a proprietary profiler that will not work on Debugging Stations but requires a Reference Tool instead – it has 256MB extra main RAM, bumping up the main system RAM in total to 512MB (plus 256MB RAM for the RSX) – which, together with the unit’s Communication Processor, gives it the extra horsepower it needs to do real-time profiling on-the-fly amongst some other things like realtime video capturing and graphical analysis through GPAD – which I’ll touch on in a moment.

LibGCM HUD


A very nice feature which seems to be only available on debug PS3s is the ability to run code with an RSX profiling tool called ‘GCM HUD’. As the name would imply, this is a Heads-Up Display overlaid on top of the application you’re running that neatly provides you with a point-and-click interface giving you access to features such as as:

  • Fragment/Vertex program debugger (with the ability to set breakpoints and step through the code line by line from the console itself – with no PC having to be involved or connected to the PS3.
  • RSX Performance Counters (telling you how effectively you’re utilizing the RSX).
  • RSX Command Buffer log (showing you the workload of the RSX in real-time)

The whole interface is mouse-driven – so to interact with it, you have to hook up an USB mouse to be able to control it. This is not where the usefulness of this tool ends, however – GPAD (Graphics Performance Analyzer and Debugger) allows you to interface your PC with this HUD and dump all the performance data to your PC. However, again, some options – such as live video capturing – can only be done with the Reference Tool.

GPAD on a Windows PC interfacing with a debug PS3 running GCM HUD

Samples

A brief rundown of the samples that struck my eye –

Deferred shading

Deferred shading lends itself well to the architecture of the PS3 where you have a comparatively humble GPU (RSX) and up to 6 SPEs each running at 3.2GHz – the idea is to essentially subdivide the result of a shading algorithm into multiple parts that can be spread across different render targets/CPUs only to combine them at the end into one composite whole. Using this approach, the RSX can simply offload a lot of the vertex and fragment computations that are done in shaders to the SPEs which in turn crunch through the calculations (the original shader algorithm having been subdivided into parts for each SPE to chew through) only for the RSX to combine all these separate parts and render the picture.

Commercial game developers like DICE have started using this approach for games like Battlefield 3 to achieve graphical results and performance which normally would have been unattainable if all that was available to them was the RSX alone. A link to a slideshow presentation is available here.

Deferred shading sample pics

Yes, the FPS count starts to drop heavily once you start adding extra spot lights.

PSGL samples

At one point, Sony was asking developers whether they would be interested in having PSGL conform to the OpenGL ES 2.0 specs (link here). This has unfortunately never happened however, as developers seem to have mostly preferred to go with libGCM as their main graphics API of choice on PS3. This has meant that the development environment has started becoming more libGCM-centric over the years with PSGL eventually becoming a second-class citizen – in fact, new features like 3D stereo mode is not even possible unless you are using libGCM directly.

However, this has not deterred them from still making available a couple of nice samples that illustrate what PSGL is capable of in conjunction with Cg shaders (which is mostly what we use with the homebrew emulators up until now). Some nice examples show Cg being applied to render hair, skin mapping, normal mapping, parallax mapping, and sophisticated water effects (the latter ones definitely being a cut above our own water.cg shader – made by Themaister).

Below you can see some screenshots illustrating nice tech demos using Cg and SPUs in tandem –

Cg being used for hair specular highlights

Cg being used for water ripple effects

Conclusion

What the debug PS3 will allow us to do is to finally start getting rid of bugs – memory leaks that were simply impossible to flush out with a retail box because of the inherent limitations of printf debugging. There are quite a few memory leaks that can now be tracked down in FBANext PS3 through crash dumps and live TTY logging.

Most ambitious of all, it will finally allow us to start writing code for SPUs – which I didn’t even want to attempt doing before because of the pain that was the retail environment. It remains to be seen whether we can really offload much if anything in the emulators to the SPUs – however, I have seen many creative uses for PPU-to-SPU offloading in some samples already, including function/virtual function call offloading to specific SPEs amongst a host of other things I didn’t even consider before.

Expect to see some real solid progress on MAMENext PS3 very soon, and for the emulators to improve immeasurably overall. Still, there are some inherent limitations even to the Debugging Station – the proprietary profiler will only allow me to do static analysis right now with a Debugging Station – if we wanted real-time profiling, we would have to be in possession of a Reference Toolkit, which is simply above anybody’s budget to be honest. This means that it might be impossible for any homebrew dev to ever do a serious attempt at porting a demanding emulator such as PCSX2 or Dolphin even with a debugging station because libgcov will only carry them so far – definitely forget it altogether on the retail units and the limited development environment they have available to them (whether it be the PS3 SDK or PSL1GHT).

The debugger that comes with it however is a seriously powerful tool and can carry us a long ways since it shows us live disassemblies of the code – meaning we can at least set a breakpoint somewhere, look at the ASM that the ‘shitty’ PS3 compiler turned our C/C++ code into, and then rewrite the code ad nauseam until the code generation starts to make more sense of our code and translate it to faster ASM code.

SNES9xNext – Android Release

With VBANext rolling on in the background we took a small side step and got SNES9xNext repository going. To help mark this release I prepared a port of SNES9xNext for Android. This is particularily interesting since all current SNES ports on Android are of SNES9x 1.43. The SNES9x 1.43 branch is known for its raw speed so it is a chose candidate for running on slower CPUs. Since SNES9x 1.5X the team has gone towards improving audio by adding blargg’s audio core among other things. In turn the SNES9x 1.5X branch is quite a bit slower. Putting arguments aside which is better the plan here is to streamline being able to quickly deploy each version. Currently SNES9x 1.43 runs full speed with frameskip=0 on all games on most native Android 1.6 or higher devices. While SNES9x 1.52 requires a newer tablet basically. On my HTC Incredible S which has a 1Ghz ARM-7 CPU I get ~55fps on most games with frameskip = 0 and full speed with frameksip = 1.


Download SNES9xNext 1.52 Android

Please post feedback in the forums!

SD3

Shadowrun

Enjoy!
Halsafar

Homebrew Cafe Forum opened

Homebrew Cafe Forums

The Homebrew Cafe Forum has just been opened. Hop on and register – moderation will be limited to pretty much three guys at the start, but we’ll see where it goes.

It was becoming necessary to have a forum eventually – as people have been meaning to get into contact with me through some means other than unofficial scene forums. This forum will serve a variety of purposes – there is a PS3 emulators section where users can post questions, suggestions, bug reports, requests and whatnot to do with the PS3 emulators. There is also a PC section dedicated to Themaister’s SSNES (and upcoming PC frontend ports). There are also other boards of a more slightly technical bent where porters/devs can get into contact with me or others – or simply idle tech talk if that’s what people prefer.

Android NDK r6 Released

This is a day late but none the less important to the mobile efforts.  Google released the sixth major version of its native development kit (NDK).  For those who are unaware this allows you to run C/C++ on Android devices and communicate to the Dalvik via the Java Native Interface (JNI).

This release brings us one very cool change for future tablet owners.  You can now build for x86 architecture!  Truly exciting for the next generation of tablets running on Intel chips. (http://gizmodo.com/5551822/intel-oak-trail-is-official-tablet-processors-with-windows-android-and-meego-support).

For developers this release brings a critical new piece of software for the debugging process.  Currently debugging the Java side of things is 100% solved and very easy to use.  However if you need to debug things in native land you are completely sol unless your phone manufacturer left gdb support available on it (or you can root your phone).  The NDK does come with ndk-gdb which is gdb as we all know it.  However some phones like mine (HTC Incredible S) are locked down in some nasty ways that as a side effect broke gdb.  So really all I have to go on is a stack trace which contains all hex addresses and no symbols, only which library the crash occured in.  There are ways to take the stack trace and a decompiled version of your library start to map things out.  This proved cumbersome at best.  Finally Google has released ndk-stack.  This analyzes the stack trace and spits out some meaningful information.

ndk-stack converts this classic stack dump:

I/DEBUG   ( 1272): *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
I/DEBUG   ( 1272): Build fingerprint: 'sasktel_canada_wwe/htc_vivo/vivo/vivo:2.2.1/FRG83D/339842:user/release-keys'
I/DEBUG   ( 1272): pid: 17249, tid: 17273  >>> ca.halsafar.foo <<<
I/DEBUG   ( 1272): signal 11 (SIGSEGV), fault addr 00000000
I/DEBUG   ( 1272):  r0 00000000  r1 80d93b8c  r2 80d94b8c  r3 00008928
I/DEBUG   ( 1272):  r4 80d7d2b0  r5 000168d8  r6 80d94b8c  r7 80d93ab0
I/DEBUG   ( 1272):  r8 00008928  r9 44224f40  10 80d7d378  fp 49e94ed8
I/DEBUG   ( 1272):  ip 00000000  sp 49e94b28  lr 80d2a488  pc 80d516dc  cpsr a0000010
I/DEBUG   ( 1272):  d0  6843646565707361  d1  2f76654464696f6e
I/DEBUG   ( 1272):  d2  65747461626d6167  d3  2e2e2f2f696e6a65
I/DEBUG   ( 1272):  d4  626d616762696c2f  d5  6372732f65747461
I/DEBUG   ( 1272):  d6  2e79726f6d656d2f  d7  293935343a707063
I/DEBUG   ( 1272):  d8  4397000041f00000  d9  43e0000000000000
I/DEBUG   ( 1272):  d10 41dfffffffc00000  d11 c3e0000000000000
I/DEBUG   ( 1272):  d12 0000000000000000  d13 0000000000000000
I/DEBUG   ( 1272):  d14 0000000000000000  d15 0000000000000000
I/DEBUG   ( 1272):  d16 3fe40fd000000000  d17 3e23b592a81fa000
I/DEBUG   ( 1272):  d18 3fe0000000000000  d19 3fe00000013b592b
I/DEBUG   ( 1272):  d20 003f003f003f003f  d21 003f003f003f003f
I/DEBUG   ( 1272):  d22 0006000300070002  d23 0006000300070002
I/DEBUG   ( 1272):  d24 0101010101010101  d25 0000000000000000
I/DEBUG   ( 1272):  d26 0000000000000000  d27 0000000000000000
I/DEBUG   ( 1272):  d28 0000000000000000  d29 0000000000000000
I/DEBUG   ( 1272):  d30 001f001f001f001f  d31 001f001f001f001f
I/DEBUG   ( 1272):  scr 20000012
I/DEBUG   ( 1272):
I/DEBUG   ( 1272):          #00  pc 000516dc  /data/data/ca.halsafar.foo/lib/libfoo.so
I/DEBUG   ( 1272):          #01  pc 000171f4  /system/lib/libdvm.so
I/DEBUG   ( 1272):          #02  pc 00045a78  /system/lib/libdvm.so
I/DEBUG   ( 1272):          #03  pc 0004b25a  /system/lib/libdvm.so
I/DEBUG   ( 1272):          #04  pc 0001c160  /system/lib/libdvm.so
I/DEBUG   ( 1272):          #05  pc 00022ed4  /system/lib/libdvm.so
I/DEBUG   ( 1272):          #06  pc 00021d70  /system/lib/libdvm.so
I/DEBUG   ( 1272):          #07  pc 0005d860  /system/lib/libdvm.so
I/DEBUG   ( 1272):          #08  pc 0005da90  /system/lib/libdvm.so
I/DEBUG   ( 1272):          #09  pc 000507cc  /system/lib/libdvm.so
I/DEBUG   ( 1272):          #10  pc 00011014  /system/lib/libc.so
I/DEBUG   ( 1272):          #11  pc 00010b00  /system/lib/libc.so
I/DEBUG   ( 1272):
I/DEBUG   ( 1272): code around pc:
I/DEBUG   ( 1272): 80d516bc a0600002 b3a00000 e28550d8 e7840005
I/DEBUG   ( 1272): 80d516cc e28710dc e1a02006 e1a03008 e1a0000c
I/DEBUG   ( 1272): 80d516dc e59cc000 e1a0e00f e59cf00c e59f1054
I/DEBUG   ( 1272): 80d516ec e1a03080 e59f0050 e3a02b5a e282c0c8
I/DEBUG   ( 1272): 80d516fc e784300c e08f2000 e08f1001 e3a00003
I/DEBUG   ( 1272):
I/DEBUG   ( 1272): code around lr:
I/DEBUG   ( 1272): 80d2a468 e1a01082 ebffffb9 e1a06000 e5940000
I/DEBUG   ( 1272): 80d2a478 e3a01a05 e2813030 e7901003 eb0009e3
I/DEBUG   ( 1272): 80d2a488 e3560000 a04060c6 e5850000 e1a00006
I/DEBUG   ( 1272): 80d2a498 e8bd8070 e92d4010 e1a04000 e5900000
I/DEBUG   ( 1272): 80d2a4a8 e3500000 0a000002 eb0020df e5940000
I/DEBUG   ( 1272):
I/DEBUG   ( 1272): stack:
I/DEBUG   ( 1272):     49e94ae8  80d93ab0
I/DEBUG   ( 1272):     49e94aec  80d7d378  /data/data/ca.halsafar.foo/lib/libfoo.so
I/DEBUG   ( 1272):     49e94af0  44224f40
I/DEBUG   ( 1272):     49e94af4  80d7d378  /data/data/ca.halsafar.foo/lib/libfoo.so
I/DEBUG   ( 1272):     49e94af8  49e94ed8
I/DEBUG   ( 1272):     49e94afc  80d33d78  /data/data/ca.halsafar.foo/lib/libfoo.so
I/DEBUG   ( 1272):     49e94b00  0026a148  [heap]
I/DEBUG   ( 1272):     49e94b04  80d93b84
I/DEBUG   ( 1272):     49e94b08  80db9088
I/DEBUG   ( 1272):     49e94b0c  80d93b84
I/DEBUG   ( 1272):     49e94b10  ffffffff
I/DEBUG   ( 1272):     49e94b14  80d93ab0
I/DEBUG   ( 1272):     49e94b18  80d7d2b0  /data/data/ca.halsafar.foo/lib/libfoo.so
I/DEBUG   ( 1272):     49e94b1c  000168d8  [heap]
I/DEBUG   ( 1272):     49e94b20  df002777
I/DEBUG   ( 1272):     49e94b24  e3a070ad
I/DEBUG   ( 1272): #00 49e94b28  80d93b84
I/DEBUG   ( 1272):     49e94b2c  00000000
I/DEBUG   ( 1272):     49e94b30  00300298  [heap]
I/DEBUG   ( 1272):     49e94b34  49e94b70
I/DEBUG   ( 1272):     49e94b38  00000000
I/DEBUG   ( 1272):     49e94b3c  479790a0  /dev/ashmem/mspace/dalvik-heap/2 (deleted)
I/DEBUG   ( 1272):     49e94b40  44224f48
I/DEBUG   ( 1272):     49e94b44  49e94b50
I/DEBUG   ( 1272):     49e94b48  44224f2c
I/DEBUG   ( 1272):     49e94b4c  80a171f8  /system/lib/libdvm.so
I/DEBUG   ( 1272): #01 49e94b50  0027e3f0  [heap]
I/DEBUG   ( 1272):     49e94b54  41aaecec  /dev/ashmem/dalvik-LinearAlloc (deleted)
I/DEBUG   ( 1272):     49e94b58  479790a0  /dev/ashmem/mspace/dalvik-heap/2 (deleted)
I/DEBUG   ( 1272):     49e94b5c  44224f40
I/DEBUG   ( 1272):     49e94b60  80a173c0  /system/lib/libdvm.so
I/DEBUG   ( 1272):     49e94b64  0027e3f0  [heap]
I/DEBUG   ( 1272):     49e94b68  49e94b70
I/DEBUG   ( 1272):     49e94b6c  80a45a7b  /system/lib/libdvm.so

Into this:

Stack frame #00  pc 000516dc  /data/data/ca.halsafar.foo/lib/libfoo.so: Routine step in /home/halsafar/androidDev/foo/jni//Application.cpp:469
Stack frame #01  pc 000171f4  /system/lib/libdvm.so
Stack frame #02  pc 00045a78  /system/lib/libdvm.so
Stack frame #03  pc 0004b25a  /system/lib/libdvm.so
Stack frame #04  pc 0001c160  /system/lib/libdvm.so
Stack frame #05  pc 00022ed4  /system/lib/libdvm.so
Stack frame #06  pc 00021d70  /system/lib/libdvm.so
Stack frame #07  pc 0005d860  /system/lib/libdvm.so
Stack frame #08  pc 0005da90  /system/lib/libdvm.so
Stack frame #09  pc 000507cc  /system/lib/libdvm.so
Stack frame #10  pc 00011014  /system/lib/libc.so
Stack frame #11  pc 00010b00  /system/lib/libc.so

I have not posted any information on the mobile side so far. Consider this a first step into getting into the habit of posting. The plan is to bring all the Android ports inline with the the PS3 ones.

Halsafar

VBANext – Launch

As of today, VBA PS3 has evolved into VBANext. All future development will happen on this page :

https://code.google.com/p/vbanext/

In short, VBA PS3 is dead – long live VBANext. The same will happen to SNES9x PS3 shortly when it blossoms into SNES9x Next (new name for SNES9x Slim).

Platforms


Phoenix Wright: Ace Attorney (Japanese version – Gyakuten Saiban) on VBANext (PS3 version shown here) with the dot shader.

This new project will be a faster, slimmed-down version of VBA-M that is currently aimed at three platforms:

  • PlayStation3
  • Xbox 360 (WIP)
  • Mobile (WIP)

It will replace both VBA PS3 on PlayStation3 and Lantus’ VBA360 0.03 on Xbox 360. In addition to that, a port to mobile platforms is tentatively in the works.

A few screenshots (taken from the PS3 version) can be viewed here.

PlayStation3


Donkey Kong ’94 with Super Game Boy borders on VBANext (PS3 version shown here).

As for the PS3 port – a lot of progress has been made over the past few weeks. Super Game Boy border support is now in – when you select a game that is Super Game Boy-compatible from the ROM menu, it will display the border that would be visible on a real SNES with a Super Game Boy add-on cart.

It will perhaps be possible to add fourplayer gamepad support at a later date for Super Game Boy games as well – certain games like Wario Blast made use of the Super Gameboy’s access to the host hardware (SNES) to allow for multiplayer support with regular SNES pads. VBA-M supports this out of the box – so it would be a shame to let it go to waste.

On the display front – FBO mode will be added – this will allow for two shaders to be selected at once. All the features that are currently in SNES9x PS3 and other emulators will be added as well – for instance, border support (different from the Super Game Boy border support which is built into VBA-M) and possibly game aware shaders as well.

Mega Man V – another Super Game Boy-compatible game – shown running here on VBANext (PS3 version shown).

Xbox 360

Final Fantasy Tactics Advance running on VBANext (PS3 version shown) – now sans the FPS slowdown in the introduction screen.

The Xbox 360 port will require some cleaning up. I will use Lantus’ VBA360 0.03 sourcecode at first and then try to slim it down by removing dependencies such as libSDL. I’m confident performance will be even better now than it previously was on 360 – since VBA PS3 was based on Lantus’ core code changes after all – with the new slimline core, it is only bound to get better.

Sonic Advance 1/2 will work again with this updated Xbox 360 version because of the removal of the SFML network code (note – if a porter wishes to do so – he can reimplement this again by defining the switch ‘NO_SFML’ – the SFML network code’s only purpose is to allow Dolphin – the Gamecube/Wii emulator – and VBA-M interoperability – it has no other purpose and it actually breaks these two games from working).

I also got rid of a rather annoying display bug in Advance Wars 2 for the PS3 port (now evolved into VBANext) – so obviously that will be fixed on 360 as well, since it suffered from the same problem.

Mobile

A mobile port for Android is tentatively in the works. More progress on this one will be posted shortly.