[mythtv-users] find_orphans.py

Thu May 15 01:27:27 UTC 2014

> On 15 May 2014, at 10:43 am, "Jack McGee" <jack at greendesk.net> wrote:
> 
>> On 05/14/2014 08:02 PM, Mark Perkins wrote:
>> 
>>>> On 15 May 2014, at 9:42 am, "Jack McGee" <jack at greendesk.net> wrote:
>>>> 
>>>>> On 05/13/2014 09:24 PM, Jack McGee wrote:
>>>>> On 05/13/2014 07:59 PM, Mark Perkins wrote:
>>>>> 
>>>>>>> On 14 May 2014, at 7:48 am, "Jack McGee" <jack at greendesk.net> wrote:
>>>>>>> 
>>>>>>>> On 05/13/2014 01:40 PM, Raymond Wagner wrote:
>>>>>>>> On 5/13/2014 2:06 PM, Jack McGee wrote:
>>>>>>>> I'm on Mythbuntu, 12.04, .27 and using the script:
>>>>>>>> 
>>>>>>>> python /usr/share/mythtv/find_orphans.py
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>> nothing has happened as a result. I know from experience, this can take some time, but after a day, I can't see any difference. In the mythbackend.log:
>>>>>>> You can't just run it. It's interactive. It requires you check and confirm what it wants to delete before it will actually issue those commands to the backend.
>>>>>> I did, picked a choice, answered yes. Actually tried every choice. A
>>>>>> multiple times.
>>>>>> 
>>>>>> 
>>>>>> ...
>>>>> Remember that with MythTV slow delete the delete can take minutes / hours / days depending on volume to be deleted. And I think mythbackend needs to be running while deleting. Perhaps you may have queued files to be deleted multiple times?
>>>> Thanks. I think that was it.  Took more than a day to delete recording entries, but they are gone now.  Still waiting on the other orphans, but will wait longer....
>>> 
>>> Script crashed, I will paste that below.   Went to the frontend and it removed most of my recordings, including many that were intact. I restored database from last night's backup, and I guess I will just delete the orphaned files from the frontend interface.
>>> 
>>> I have had this in crontab for years:
>>> 15 1 * * * perl /usr/share/mythtv/optimize_mythdb.pl >/dev/null 2>&1 # JOB_ID_1
>>> 
>>> 
>>> Traceback (most recent call last):
>>>  File "/usr/share/mythtv/find_orphans.py", line 231, in <module>
>>>    main()
>>>  File "/usr/share/mythtv/find_orphans.py", line 167, in main
>>>    recs, zerorecs, orphvids, orphimgs, dbbackup, unfiltered = populate(host)
>>>  File "/usr/share/mythtv/find_orphans.py", line 72, in populate
>>>    for sg in DB.getStorageGroup():
>>>  File "/usr/lib/python2.7/dist-packages/MythTV/database.py", line 1397, in getStorageGroup
>>>    yield StorageGroup.fromRaw(row, self)
>>>  File "/usr/lib/python2.7/dist-packages/MythTV/database.py", line 128, in fromRaw
>>>    cls._setClassDefs(db)
>>>  File "/usr/lib/python2.7/dist-packages/MythTV/database.py", line 82, in _setClassDefs
>>>    cls._field_order = db.tablefields[cls._table]
>>>  File "/usr/lib/python2.7/dist-packages/MythTV/database.py", line 1146, in __getitem__
>>>    with self._db.cursor(self._log) as cursor:
>>>  File "/usr/lib/python2.7/dist-packages/MythTV/connections.py", line 139, in cursor
>>>    conn = self.acquire()
>>>  File "/usr/lib/python2.7/dist-packages/MythTV/connections.py", line 93, in acquire
>>>    conn = self._connect()
>>>  File "/usr/lib/python2.7/dist-packages/MythTV/connections.py", line 114, in _connect
>>>    return dbmodule.dbconnect(self.dbconn, self.log)
>>>  File "/usr/lib/python2.7/dist-packages/MythTV/_conn_mysqldb.py", line 25, in dbconnect
>>>    charset='utf8')
>>>  File "/usr/lib/python2.7/dist-packages/MySQLdb/__init__.py", line 81, in Connect
>>>    return Connection(*args, **kwargs)
>>>  File "/usr/lib/python2.7/dist-packages/MySQLdb/connections.py", line 187, in __init__
>>>    super(Connection, self).__init__(*args, **kwargs2)
>>> _mysql_exceptions.OperationalError: (1040, 'Too many connections')
>>> mythuser at PVR1:/var/log/mythtv$ python /usr/share/mythtv/find_orphans.py
>>> Traceback (most recent call last):
>>>  File "/usr/share/mythtv/find_orphans.py", line 231, in <module>
>>>    main()
>>>  File "/usr/share/mythtv/find_orphans.py", line 167, in main
>>>    recs, zerorecs, orphvids, orphimgs, dbbackup, unfiltered = populate(host)
>>>  File "/usr/share/mythtv/find_orphans.py", line 72, in populate
>>>    for sg in DB.getStorageGroup():
>>>  File "/usr/lib/python2.7/dist-packages/MythTV/database.py", line 1397, in getStorageGroup
>>>    yield StorageGroup.fromRaw(row, self)
>>>  File "/usr/lib/python2.7/dist-packages/MythTV/database.py", line 128, in fromRaw
>>>    cls._setClassDefs(db)
>>>  File "/usr/lib/python2.7/dist-packages/MythTV/database.py", line 82, in _setClassDefs
>>>    cls._field_order = db.tablefields[cls._table]
>>>  File "/usr/lib/python2.7/dist-packages/MythTV/database.py", line 1146, in __getitem__
>>>    with self._db.cursor(self._log) as cursor:
>>>  File "/usr/lib/python2.7/dist-packages/MythTV/connections.py", line 139, in cursor
>>>    conn = self.acquire()
>>>  File "/usr/lib/python2.7/dist-packages/MythTV/connections.py", line 93, in acquire
>>>    conn = self._connect()
>>>  File "/usr/lib/python2.7/dist-packages/MythTV/connections.py", line 114, in _connect
>>>    return dbmodule.dbconnect(self.dbconn, self.log)
>>>  File "/usr/lib/python2.7/dist-packages/MythTV/_conn_mysqldb.py", line 25, in dbconnect
>>>    charset='utf8')
>>>  File "/usr/lib/python2.7/dist-packages/MySQLdb/__init__.py", line 81, in Connect
>>>    return Connection(*args, **kwargs)
>>>  File "/usr/lib/python2.7/dist-packages/MySQLdb/connections.py", line 187, in __init__
>>>    super(Connection, self).__init__(*args, **kwargs2)
>>> _mysql_exceptions.OperationalError: (1040, 'Too many connections')
>>> mythuser at PVR1:/var/log/mythtv$ python /usr/share/mythtv/find_orphans.py
>>> Traceback (most recent call last):
>>>  File "/usr/share/mythtv/find_orphans.py", line 231, in <module>
>>>    main()
>>>  File "/usr/share/mythtv/find_orphans.py", line 167, in main
>>>    recs, zerorecs, orphvids, orphimgs, dbbackup, unfiltered = populate(host)
>>>  File "/usr/share/mythtv/find_orphans.py", line 72, in populate
>>>    for sg in DB.getStorageGroup():
>>>  File "/usr/lib/python2.7/dist-packages/MythTV/database.py", line 1397, in getStorageGroup
>>>    yield StorageGroup.fromRaw(row, self)
>>>  File "/usr/lib/python2.7/dist-packages/MythTV/database.py", line 128, in fromRaw
>>>    cls._setClassDefs(db)
>>>  File "/usr/lib/python2.7/dist-packages/MythTV/database.py", line 82, in _setClassDefs
>>>    cls._field_order = db.tablefields[cls._table]
>>>  File "/usr/lib/python2.7/dist-packages/MythTV/database.py", line 1146, in __getitem__
>>>    with self._db.cursor(self._log) as cursor:
>>>  File "/usr/lib/python2.7/dist-packages/MythTV/connections.py", line 139, in cursor
>>>    conn = self.acquire()
>>>  File "/usr/lib/python2.7/dist-packages/MythTV/connections.py", line 93, in acquire
>>>    conn = self._connect()
>>>  File "/usr/lib/python2.7/dist-packages/MythTV/connections.py", line 114, in _connect
>>>    return dbmodule.dbconnect(self.dbconn, self.log)
>>>  File "/usr/lib/python2.7/dist-packages/MythTV/_conn_mysqldb.py", line 25, in dbconnect
>>>    charset='utf8')
>>>  File "/usr/lib/python2.7/dist-packages/MySQLdb/__init__.py", line 81, in Connect
>>>    return Connection(*args, **kwargs)
>>>  File "/usr/lib/python2.7/dist-packages/MySQLdb/connections.py", line 187, in __init__
>>>    super(Connection, self).__init__(*args, **kwargs2)
>>> _mysql_exceptions.OperationalError: (1040, 'Too many connections')
>>> 
>>> _______________________________________________
>> Someone who knows how it really works will comment shortly but I think you ran out of connections to the database.
>> 
>> What distribution are you running? Do you have a /etc/mysql/conf.d/mythtv.cnf file, if yes what is the value of the max_connections parameter. If you don't have that file (or parameter in that file) you may need to check /etc/mysql/my.cnf
>> 
>> I believe MySQL default value is 100 but recommendation is to increase this to 250 or so by adding a mythtv.cnf file, see about 3rd post here http://www.gossamer-threads.com/lists/mythtv/users/554094.
> 
> 
> Thanks, that parameter is not in the file.  I only have 2 gig ram, and 32 bit flavor of Ubuntu, but am planning upgrade to new motherboard, more ram, and 64 bit Ubuntu and will try it there.
> 
> Until last weekend, btw, this was running on 1 gig ram and I know I had run that script on that a few times with no issues.
> 
> 
> _______________________________________________

You could try mysqltuner.pl from 5th post in that link and see how many connections you have hit to confirm if that is the problem. I don't think increasing RAM or going to 64bit will change anything if you have hit the set connection limit, well the limit is the limit I guess. There are probably only limited times where you have that many connections (ie I don't suppose you run find_orphans.py daily).