[mythtv-commits] Ticket #7083: Segfault in VideoOutputXv::PrepareFrameMem() with certain aspect ratios.

Thu Sep 17 04:25:51 UTC 2009

#7083: Segfault in VideoOutputXv::PrepareFrameMem() with certain aspect ratios.
-------------------------------------+--------------------------------------
 Reporter:  myth at jstenback.com       |       Owner:  janne  
     Type:  defect                   |      Status:  new    
 Priority:  major                    |   Milestone:  unknown
Component:  MythTV - Video Playback  |     Version:  head   
 Severity:  medium                   |     Mlocked:  0      
-------------------------------------+--------------------------------------
 When running mythfrontend in a window in a vnc session (for testing
 purposes), I noticed that with certain window sizes I get a segfault in
 VideoOutputXv::PrepareFrameMem(). The problem appears to be related to the
 allocation of the sbuf buffer in that method, if I double the size, the
 segfault goes away, but I get skewed video output (the bottom left corner
 is shifted over to being the bottom right corner, at a 45 degree angle, so
 something seems to be off by one pixel per line or somesuch). The
 allocation in question is this line:

 unsigned char *sbuf = new unsigned char[
     display_visible_rect.width() * display_visible_rect.height() * 3 / 2];

 The segfault is from deleting that buffer later on. I see the problem when
 I run mythfrontend in a window with the size 711x400px, if I change the
 size to 712x400, the segfault goes away.

 The mythtv version is as follows:

 MythTV Version   : 21870M
 MythTV Branch    : trunk
 Network Protocol : 48
 Library API      : 0.22.20090912-2
 QT Version       : 4.5.2
 Options compiled in:
  linux release using_oss using_alsa using_backend using_dvb using_frontend
 using_hdhomerun using_hdpvr using_iptv using_ivtv using_joystick_menu
 using_lirc using_mheg using_opengl_video using_opengl_vsync using_qtwebkit
 using_v4l using_x11 using_xrandr using_xv using_xvmc using_xvmc_vld
 using_xvmcw using_bindings_perl using_bindings_python using_opengl
 using_ffmpeg_threads using_live using_mheg

 This is all using the standard CPU+ playback profile.

 Valgrind had this to say about the segfault:

 ==14485==
 ==14485== Thread 11:
 ==14485== Invalid write of size 1
 ==14485==    at 0x6D28E3A: swScale_MMX2 (swscale.c:536)
 ==14485==    by 0x6D1E4B7: sws_scale (swscale.c:3206)
 ==14485==    by 0x56BAB40: VideoOutputXv::PrepareFrameMem(VideoFrame_*,
 FrameScanType) (videoout_xv.cpp:2207)
 ==14485==    by 0x56BB4E3: VideoOutputXv::PrepareFrame(VideoFrame_*,
 FrameScanType) (videoout_xv.cpp:2258)
 ==14485==    by 0x554E9BB: NuppelVideoPlayer::AVSync()
 (NuppelVideoPlayer.cpp:2489)
 ==14485==    by 0x5552DD8: NuppelVideoPlayer::DisplayNormalFrame()
 (NuppelVideoPlayer.cpp:2946)
 ==14485==    by 0x5553E22: NuppelVideoPlayer::OutputVideoLoop()
 (NuppelVideoPlayer.cpp:3085)
 ==14485==    by 0x5553F9D:
 NuppelVideoPlayer::kickoffOutputVideoLoop(void*)
 (NuppelVideoPlayer.cpp:3163)
 ==14485==    by 0x99EF3D9: start_thread (in /lib64/libpthread-2.9.so)
 ==14485==    by 0xC38D2BC: clone (in /lib64/libc-2.9.so)
 ==14485==  Address 0x2e0eadb0 is 0 bytes after a block of size 426,600
 alloc'd
 ==14485==    at 0x4C2561C: operator new[](unsigned long)
 (vg_replace_malloc.c:274)
 ==14485==    by 0x56BA604: VideoOutputXv::PrepareFrameMem(VideoFrame_*,
 FrameScanType) (videoout_xv.cpp:2176)
 ==14485==    by 0x56BB4E3: VideoOutputXv::PrepareFrame(VideoFrame_*,
 FrameScanType) (videoout_xv.cpp:2258)
 ==14485==    by 0x554E9BB: NuppelVideoPlayer::AVSync()
 (NuppelVideoPlayer.cpp:2489)
 ==14485==    by 0x5552DD8: NuppelVideoPlayer::DisplayNormalFrame()
 (NuppelVideoPlayer.cpp:2946)
 ==14485==    by 0x5553E22: NuppelVideoPlayer::OutputVideoLoop()
 (NuppelVideoPlayer.cpp:3085)
 ==14485==    by 0x5553F9D:
 NuppelVideoPlayer::kickoffOutputVideoLoop(void*)
 (NuppelVideoPlayer.cpp:3163)
 ==14485==    by 0x99EF3D9: start_thread (in /lib64/libpthread-2.9.so)
 ==14485==    by 0xC38D2BC: clone (in /lib64/libc-2.9.so)
 ==14485==
 ==14485== Invalid read of size 4
 ==14485==    at 0x6D2E720: yuv420_rgb16_MMX2 (yuv2rgb_template.c:193)
 ==14485==    by 0x6D1E4B7: sws_scale (swscale.c:3206)
 ==14485==    by 0x530906B: myth_sws_img_convert(AVPicture*, PixelFormat,
 AVPicture*, PixelFormat, int, int) (myth_imgconvert.cpp:54)
 ==14485==    by 0x56BAC01: VideoOutputXv::PrepareFrameMem(VideoFrame_*,
 FrameScanType) (videoout_xv.cpp:2219)
 ==14485==    by 0x56BB4E3: VideoOutputXv::PrepareFrame(VideoFrame_*,
 FrameScanType) (videoout_xv.cpp:2258)
 ==14485==    by 0x554E9BB: NuppelVideoPlayer::AVSync()
 (NuppelVideoPlayer.cpp:2489)
 ==14485==    by 0x5552DD8: NuppelVideoPlayer::DisplayNormalFrame()
 (NuppelVideoPlayer.cpp:2946)
 ==14485==    by 0x5553E22: NuppelVideoPlayer::OutputVideoLoop()
 (NuppelVideoPlayer.cpp:3085)
 ==14485==    by 0x5553F9D:
 NuppelVideoPlayer::kickoffOutputVideoLoop(void*)
 (NuppelVideoPlayer.cpp:3163)
 ==14485==    by 0x99EF3D9: start_thread (in /lib64/libpthread-2.9.so)
 ==14485==    by 0xC38D2BC: clone (in /lib64/libc-2.9.so)
 ==14485==  Address 0x2e0eadb0 is 0 bytes after a block of size 426,600
 alloc'd
 ==14485==    at 0x4C2561C: operator new[](unsigned long)
 (vg_replace_malloc.c:274)
 ==14485==    by 0x56BA604: VideoOutputXv::PrepareFrameMem(VideoFrame_*,
 FrameScanType) (videoout_xv.cpp:2176)
 ==14485==    by 0x56BB4E3: VideoOutputXv::PrepareFrame(VideoFrame_*,
 FrameScanType) (videoout_xv.cpp:2258)
 ==14485==    by 0x554E9BB: NuppelVideoPlayer::AVSync()
 (NuppelVideoPlayer.cpp:2489)
 ==14485==    by 0x5552DD8: NuppelVideoPlayer::DisplayNormalFrame()
 (NuppelVideoPlayer.cpp:2946)
 ==14485==    by 0x5553E22: NuppelVideoPlayer::OutputVideoLoop()
 (NuppelVideoPlayer.cpp:3085)
 ==14485==    by 0x5553F9D:
 NuppelVideoPlayer::kickoffOutputVideoLoop(void*)
 (NuppelVideoPlayer.cpp:3163)
 ==14485==    by 0x99EF3D9: start_thread (in /lib64/libpthread-2.9.so)
 ==14485==    by 0xC38D2BC: clone (in /lib64/libc-2.9.so)

 --14485-- VALGRIND INTERNAL ERROR: Valgrind received a signal 11 (SIGSEGV)
 - exiting
 --14485-- si_code=80;  Faulting address: 0x0;  sp: 0x410D46DD0

 Attached is also the output of:

   mythfrontend -w -geometry 711x400 -v playback

 of me attempting to play back a recording (720p H264, from an HD-PVR).

 Let me know if I can provide any additional information.

-- 
Ticket URL: <http://svn.mythtv.org/trac/ticket/7083>
MythTV <http://www.mythtv.org/>
MythTV