There is a lot of documentation available online about how to auto-transcode to XviD or DivX using a single pass, but these options are no longer favourable. A single pass will always produce inferior results, and H.264 allows for good quality at a much smaller file size than XviD or DivX. With H.264 I am able to make a transcode that is about one sixth of the size of the original source, and it actually looks better than the original. I wrote this because I had to figure it out myself and I couldn't find any decent documentation anywhere online.
Updated!
This tutorial will teach you how to create a custom userjob in MythTV that will automatically remove commercials, run a multi-pass
transcode, delete the original recording, and update its database entry to point to the transcoded file upon completion. You will
then be able to watch and delete your transcoded shows through the recordings menu of your frontend. Please note that this method breaks the Direct Download and ASX Stream
features of MythWeb for any transcoded files.
Updated Again!
Now this works with multiple jobs running at the same time, with increased niceness, and now defaults to using all your processor cores.
This information is specific to Debian GNU/Linux, and has only been tested on a Debian Lenny and Debian Squeeze backend, but it should work fine on any Debian derivative, and with minor adjustments, on any modern Linux distribution.
Let's get started...
next, download my Bourne shell script (above).
First, stop mythbackend.
All you need to change in the above command is /path/to/script
That's it! Now when you schedule a recording, you can go to the postprocessing options, uncheck transcode (if it's checked), and check User Job#1 to tell it to run the script after recording/flagging commercials.
It's important to note that if the transcoding process fails in any way, you will lose your original recording, so FOLLOW THIS TUTORIAL AT YOUR OWN RISK!
If you want to set this transcoding job on by default for all newly scheduled recordings, in mythfrontend go to: TV -> TV Settings -> General -> [press "next" twice]...
at the bottom where it says "Default Job Queue Settings for
New Scheduled Recordings", check the box next to your new "Transcode to H.264" userjob.
You might also want to modify the mencoder filter chain to your liking, as it is currently optimized for a Hauppauge PVR-150 interlaced video source. As well you may need to change the audio delay in the mkvmerge command if you are getting audio and video that's out of sync. Read the comments in the Bourne shell script for more info. If you need help with mencoder, the man pages are very helpful, and there is loads of documentation available online.
Here is an alternate version of the script that can be used for transcoding shows which are broadcast in false-widescreen. It will convert the false-widescreen into real widescreen, removing all black bars around the picture. Your resulting file will be optimized for viewing on a widescreen display. This may not be necessary anymore because newer versions of Mythfrontend are really good at auto-zooming false-widescreen videos on-the-fly, but you might want to use this script if you use a different frontend, an old version of MythTV, or if you plan on using your transcoded files for other purposes.
Enjoy!
Was this tutorial helpful? Toss me a few bucks if you're feeling generous.
Comments
2010-05-20 09:57:31 EST
Subject: Audio
Comment:
This looks extremely promising, but I do have a couple of quick questions.
- Will the ogg vorbis audio keep the 5.1 sound, or will this
downmix it to 2.0 stereo?
- Have you tried any Progressive
recordings? I have a bunch that are 60fps Progressive and i am curious how
well mencoder handles them.
2010-05-27 16:13:08 EST
Subject: re: Audio
Comment:
Hi sdsnyr94, sorry it took me so long to get back to you.
I have looked around for more information on oggenc, and from what I've read,
all the channels should be preserved. However, I don't have a 5.1 system
to test it on, so I can't give you a definitive answer on that one. If you
try it out, could you please comment back and let me know if it
preserves all the channels?
For your second question, mencoder has about a zillion different filters for
every purpose. If you're just wanting to re-encode your progressive
videos to h.264 and keep them progressive (mencoder works well for this),
all you have to do is remove the yadif filter from the two mencoder
video passes in my script (remove the word yadif, from line 44,
and also from line 47). Yadif is only useful if you're re-encoding
interlaced content to progressive. For more info about mencoder, search
through the extremely long man pages (type /searchterm to search for a
certain phrase while in the man pages or ?searchterm to search upward, as
it's impossible that anyone would have enough time to read the whole
mencoder manual).
Let me know if you have any more questions.
2010-09-25 20:57:11 EST
Subject: Can't find encoded file
Comment:
Tried running this script today. It seemed to encode the file (took about
14 hours) but I can't find the encoded file.
I am using mythtv
0.23, I set the temp directory as /tmp. Looking at mythbackend.log it seems
to have written the file to the /tmp directory but can't find
it.
Here are some entries from mythtvbackend.log:
< EDIT: LOG FILE OUTPUT REMOVED DUE TO IRRELEVANCE >
Am I doing
something wrong?
2010-06-01 13:15:33 EST
Subject: re: Can't find encoded file
Comment:
Hi aussieW
I think what you have done wrong (and I'm sorry the
tutorial wasn't clear enough on this point), is you have set the output
directory to the same place as the temp working directory.
See this instruction:
"
Next to "User Job#1 Command", type
this (filling in the appropriate values for your
setup):
/path/to/bash-script/mythtv-transcode-h264.sh
"%DIR%/%FILE%" "/path/to/output-dir/%TITLE% -
%PROGSTARTISO%.mkv" "%CHANID%" "%STARTTIME%"
Make
sure that the output directory you enter in the above command exists and is
writeable by the mythtv user in the same way you did with the temp
directory. All you need to change in the command above is
/path/to/bash-script and /path/to/output-dir
"
well,
path/to/output-dir needs to be a separate folder (different from the temp
working dir), which already exsists, and with write permissions for the
mythtv user.
Note that everything is erased from the temp dir at the end of the script,
so if you save the transcoded file into the temp dir, it will be erased
along with everything else.
I hope that solves your problem. Let me know if you
need more help.
2010-06-01 15:04:52 EST
Subject: re: Can't find encoded file
Comment:
Thanks, that worked.
I noticed that the adds are still in the
encoded file. Is there a way to pass a cutlist to the encoder?
2010-06-01 17:32:55 EST
Subject: re: Can't find encoded file
Comment:
Hi,
There are three commands in my script that deal with generating
and using the cutlist.
see lines 33-35 in the
script:
"
# remove commercials
$INSTALLPREFIX/mythcommflag -c
"$3" -s "$4" --gencutlist
$INSTALLPREFIX/mythtranscode --chanid
"$3" --starttime "$4" --mpeg2
--honorcutlist
$INSTALLPREFIX/mythcommflag --file "$1"
--rebuild
"
This process assumes that your commercials have
already been flagged with the MythTV commercial flagger. It looks at the
flag list, and uses that to generate the cutlist.
Your commercials
are not being removed most likely for one of three reasons:
1. You
have not set up MythTV to auto-flag commercials when recording completes.
You should make sure to enable this (or you can enable MythTV to flag
commercials while recording is in progress).
or
2. In the
script, on line 22, INSTALLPREFIX="/usr/bin" needs to be set to the
location of your MythTV binaries. They may be located somewhere else on
your system, such as in "/usr/local/bin". Find your binaries, and make
sure INSTALLPREFIX points to the directory they are located in.
(Hint:
Type locate mythcommflag in a terminal or if that doesn't work try the
same command but type sudo at the beginning. This should show you where
your binaries are installed.)
or
3. It could be a combination
of case 1 and 2.
I hope these suggestions work for you. Let me know.
2010-06-02 07:36:51 EST
Subject: re: Can't find encoded file
Comment:
Hi
Thanks for your response. I have satisfied both case 1 and 2 so
must be something else. I have only tried on one file so far so I shall try
another and see what happens.
2010-06-02 14:55:22 EST
Subject: re: Can't find encoded file
Comment:
Hi
I've just gone back and had a look at the encoded file and it
seems that it has just cut out the commercials that were flagged by myth.
It seems that a few may have been missed by mythcommflag so that it why
they are showing in the encoded file. So it seems we are all good.
2010-09-25 20:59:24 EST
Subject: Audio Failed
Comment:
Thanks for a great tutorial.
My first test produced a very good quality
video - but no sound.
I examined the log file (see below) and didnt see
anything significant.
Any ideas what the issue is?
< EDIT: LOG FILE OUTPUT REMOVED DUE TO IRRELEVANCE >
2010-08-09 12:09:36 EST
Subject: Audio Failed
Comment:
Sorry my problem - finger trouble.
The sound is in the transcoded
file!!
(I was testing the resultant transcoded file with VLC, and MythTV
frontend had grabbed the Audio - so there was no audio from VLC!)
2010-08-10 12:21:11 EST
Subject: Re: Audio Failed
Comment:
No worries. I think I have to stop being lazy and make this comments
section collapse in some way, as it's getting really
looooooong.
:)
I'm glad no-one's having any real trouble
making this method work for them. I'm just hoping one day MythTV will
have a better version of this idea implemented in it, so people don't
have to use custom userjobs and scripts. At least some sort of built in
h.264 transcoder would be really nice. I don't know why they don't
have it yet, when it's obvious to everyone that h.264 is far superior to
xvid or divx.
2010-09-05 08:10:48 EST
Subject: db entries stay in database
Comment:
hello.
first of all thanks for this nice script.
one thing that
didn't work for me is the deletion of the database entries.
the
mpg-file gets deleted but not the database entry.
any
ideas?
greetings
rasda
2010-09-05 16:17:49 EST
Subject: works great
Comment:
I've been using your script for a while & it works great! The issues that
do exist are with mencoder itself. BIG thanks for posting this!
2010-09-06 08:31:56 EST
Subject: update
Comment:
hey. according to my db-issue it just turned out that i had put a wrong
mysql password. so forget about this.
i had one problem with "german
umlaute" like öäüß. the resulting files were always cut after an
"umlaut". so when i had lets say 4 parts of a series having an umlaut in
the %TITLE% they were overwritten.
i also tested another encoder
(handbrake) which works better (faster) for me and using handbrake also
fixed the umlaut-issue. if you are interested in my changed script let me
know.
cheers and thx
rasda
2010-09-06 16:39:01 EST
Subject: Re: update
Comment:
Hey, thanks for pointing out the issue with international characters. I
have never worked with them before, so I had no idea that they might cause
a problem in my script. I will look into a fix for this, but if anyone has
any ideas on how to fix the problem and continue using mencoder, I would
really appreciate some input. In the meantime, I would love to see your
script, as maybe it will give me some ideas on how to fix the problem.
2010-09-06 16:49:49 EST
Subject: Re: works great
Comment:
Thanks for the positive feedback kr0nk!
Yeah, mencoder is great, but
sometimes a new version will introduce new problems, while fixing older
problems. Such is life for a program which the developers aren't being
paid enough to develop.
2010-09-07 13:31:46 EST
Subject: Re: Re: update
Comment:
here is my version of the
script.
http://pastebin.com/fS3bpMpB
handbrakecli is about
one third faster than mencoder on my machine. the quality is the same i
would say. i dont know where the "umlaut" problem came from. however with
this script its working.
greetings
rasda
2010-09-07 13:47:46 EST
Subject: link and publishing of changed script
Comment:
hey.
is it okay when i publish my version with links to your version
in my blog?
rasda
2010-09-07 17:13:33 EST
Subject: Re: link and publishing of changed script
Comment:
Hi rasda,
Thank you for the script! You can link to my tutorial in
whole, but not directly to my script download link. Please give credit to
my screen name if you link. I look forward to seeing your blog entry, send
me a link when it's done.
2010-09-08 11:47:23 EST
Subject: Re: Re: link and publishing of changed script
Comment:
here it
is
https://rasda.de/blog/2010/09/08/mythtv-userjob-handbrake-transcode-convert
2010-09-22 00:50:36 EST
Subject: Has the script been modified so video will play on Nexus One cell phone?
Comment:
I used the script as is to convert a recording but it would not stream or
play on the phone when I clicked on it in the web browser. Also was
wondering about the converted video file extension being "mkv" instead of
"mp4".
2010-09-22 19:50:50 EST
Subject: Re: Has the script been modified so video will play on Nexus One cell phone?
Comment:
Hi David,
The resulting .mkv file will only work on devices that
support the Matroska container and h.264 codec.
.mp4 files are usually
made with the divx or xvid codecs.
If you want to make .mp4 files,
you will need to modify the script yourself. I am not personally interested
in .mp4 files, because xvid and divx codecs aren't as efficient as
h.264.
Alternately, you can turn to a search engine, there are
already well documented easy methods of making a MythTV userjob to auto
transcode to DIVX or XVID. These methods will probably give you a .avi
file, but it shouldn't be too hard to put the video in an .mp4 container
instead.
2010-09-22 23:57:16 EST
Subject: Re: Has the script been modified so video will play on Nexus One cell phone?
Comment:
The native video player in the Nexus One is H264. It maybe the Matroska
container that it does not know how to deal with. So I wonder if I changed
from mkvmerge to mplex if it would work. It is worth a try.
2010-09-23 13:47:59 EST
Subject: Re: Has the script been modified so video will play on Nexus One cell phone?
Comment:
I have only ever made H.264 videos in the .mkv container because I only
play the files on computers, but it should be possible to make them in the
.mp4 container (at least Wikipedia says so). Let me know if you figure it out.
2010-09-23 19:50:49 EST
Subject: Re: Has the script been modified so video will play on Nexus One cell phone?
Comment:
I have it working now using ffmpeg, but I'm still tweaking it. Right now
it worked well using wifi but the bit rate was too high for G3 so I'm
still running experiments. Send me an email and I can send you the results
once I determine the right bit rates.
2010-09-24 20:34:48 EST
Subject: Re: Has the script been modified so video will play on Nexus One cell phone?
Comment:
Pasting of script to web destroyed script by removing back slashes and
other characters. Script can be grabbed from
here:
http://www.highley-recommended.com/multimedia/mythtv-transcode-h264.sh
Sorry
for messing up this blog page.
2010-09-25 20:51:04 EST
Subject: Re: Has the script been modified so video will play on Nexus One cell phone?
Comment:
No worries, I'll clean it and leave the post with the link. Yeah, the
comment section was made by me, and has the emphasis set on not destroying
the actual tutorial page. Therefore, it might sanitize bash scripts or
other commands, making them not work at all in practice. Links are nicer
anyway, because the comments section tends to get too long.
Thank
you for adapting my script to work with cell phones! I'm always glad to
see it used for different things.
2010-09-27 10:54:37 EST
Subject: Re: Has the script been modified so video will play on Nexus One cell phone?
Comment:
I should have renamed the script so as not to cause confusion. The new name
is:
http://www.highley-recommended.com/multimedia/mythtv-transcode-nexus1-h264.sh
After
more experimenting I'm not sure of the value of transcoding to something
that will stream over 3G. I can also see that a user may need many scripts
depending on target device and streaming needs. Streaming over wifi was
definitely ok, but 3G is very problematic.
2010-11-21 06:11:29 EST
Subject: Strange file names
Comment:
Hi,
Thanks for creating the script.
It works fine,
but....
1.Since I 'm recording from DVB-C two pass encoding is not
really necessary. So I switched to one pass encoding, which saves about
half the time without any noticeable loss of quality.
2.The files the
script creates do get names that read very different under Linux and
Windows.
For example the file with a name that reads under Linux as
Test1-2010-11-21T11:15:00.MKV has under Windows the following name:
TRNG6B~V.MKV. It is the same file on the same position on the same
mythtv-server. After copying the file to a Windows machine, the file name
still reads: TRNG6B~V.MKV.
The setup is a mythtv-backend server
(Mythbuntu 10.04) and two windows XBMC-frontend clients. Because a wireless
54Mbps network isn't fast enough for serving two frontends simultaneously
using DVB-mpeg streams, I went looking for a H264 solution (same quality
and less need for bandwidth). Your script does help me almost all the way,
but the file names are causing some headaches.
Switching to the
HandBrakeCLI variant did solve the described file naming problem. But
introduced a new problem: the output file contains only sound and no
pictures. Using HandBrakeCLI manually works fine however. HandBrake and
Mencoder are both encoding 7 to 8 fps. No difference in speed to be
noticed.
3.How can the transcoded files be inserted into the
mythtv-database? After deleting the original recording, it would be nice if
the transcoded file shows up in place. At the moment I have to copy the
transcoded/exported files to my frontend. Otherwise my server can switch
off during watching via VLC, because of reaching the end of it's 15
minutes idle wait time. Not surprisingly this has a very low waf (wife
acceptance factor).
Hopefully you have some suggestions for solutions or
places to look for solutions.
Regards.
2010-11-25 17:12:21 EST
Subject: Re: Strange file names
Comment:
Hi Dutchy,
Thank you for testing my tutorial on Windows. The
filenames problem was due to the NTFS filesystem not allowing colons (:) in
filenames. I have updated my tutorial, changing the second argument of the
UserJob#1 Command from: "/path/to/output-dir/%TITLE% -
%PROGSTARTISO%.mkv" to: "/path/to/output-dir/%TITLE% - %PROGSTART%.mkv"
(notice I dropped the ISO). This change will allow the output filename to
be compatible with Windows, and to display correctly over a network share,
and in XBMC.
As for #3, I have wanted to do that for quite some
time, I just never got around to it. I'll look into adding that
functionality to my script in the next few days, and probably release an
updated version of the script shortly. Check back soon for the updated
version.
Thanks again for reporting the Windows filename bug. It
should be fixed now.
2010-11-25 22:55:51 EST
Subject: Update
Comment:
As promised, I have updated this page and the shell script so that instead
of dropping the transcoded file into a new directory and deleting the
database entry, it will drop the transcoded file into the same directory
that the original was in, and update the old database entry to point to the
newly transcoded file. Note that the original recording is now deleted upon
completion by default, whether or not the transcoding was successful. As
such, you should use small unimportant test recordings while you are trying
out the script, until you make sure everything works properly. I'm not
responsible if you accidentally lose an important recording while using my
script.
2011-04-13 14:38:24 EST
Subject: Cheap Car Hire Alicante <-- a subject line is no place for an advertisement.
Comment:
Hey, I am checking this blog using the phone and this appears to be kind of
odd. Thought you'd wish to know. This is a great write-up nevertheless,
did not mess that up.
- David
2011-02-05 18:05:07 EST
Subject: Transcode failed, then deleted file
Comment:
I couldn't get this to work. It reported
"2011-02-06 14:11:16.074
ProgramInfo(The_Lost_Room_-_20110206001500.mkv), Error: GetPlaybackURL:
'The_Lost_Room_-_20110206001500.mkv' should be local, but it can not be
found."
File "20110206001500.mpg" did exist, but was deleted when
the error was reported.
I'm not game to do any more testing for
fear of losing more files.
I'm using mythtv 0.24
2011-02-07 10:56:45 EST
Subject: Re: Transcode failed, then deleted file
Comment:
Hence why I warned everyone to test on small unimportant recordings until
you make sure it works properly. You haven't given me nearly enough info
for me to pinpoint the problem, but seeing as no-one else has had any
similar trouble so far, I would suggest re-reading my instructions
carefully if you ever decide to give it another try. If you want more help,
you could post the relevant section of your mythbackend.log file (not the
whole thing please), and your script (complete with your modifications).
Then I might be able to give you a hint as to why it's failing for you.
2011-03-30 10:31:26 EST
Subject: file won't play back
Comment:
First of all I would like to say thank you for all your work on this
transcoding to H.264. The problem I'm having is after it transcodes the
file and puts it back in the same Dir(~/var/lib/mythtv/livetv). I go to
play recording it just crashes and brings me back to main menu in mythtv.
When I look for the file ~/var/lib/mythtv/livetv/ I get a new folder 9 and
inside that is my recorded H.264 file. If I drag the file out to livetv
folder it plays from watch recorded program in mythtv. I would be very
thankful for any help. If you can't tell I'm new to linux and mythtv.
Only been running mythtv for about 3 weeks.
2011-03-30 12:31:22 EST
Subject: transcoded file don't play
Comment:
This line ($INSTALLPREFIX/mythcommflag --file "$1"
--rebuild
") is
missing from your down loadable script Do I need to add it back in? Just
check files again its now transcoding the file and putting it in the right
folder but not deleting the mepeg2 recording and png file still won't play
from mythtv tv recordings.
2011-03-30 19:35:43 EST
Subject: Re: transcoded file don't play
Comment:
Hi lowrider, the line you mentioned was from an earlier version of the
script. It has been removed because it's no longer needed. You shouldn't
add it back.
Lines 64 and 65 of the script should change the MySQL
database entry for your recording to point to the newly transcoded file,
and line 66 should remove the original mpeg2 file... This all works fine on
my system.
You mentioned that the png file won't play from the
mythtv recordings, I hope you meant the .mkv file (the mkv is the
transcoded video). If you're getting a png file in your mythtv recordings
menu, then you haven't configured the script correctly (the .png is just a
thumbnail).
I just double-checked to see if the script works, and it
works on my system if you follow all the above instructions correctly. I
would suggest you start over with a fresh copy of the script, follow the
instructions exactly, and if you still have problems, you can come back and
post your modified version of the script, along with the relevant section
of your mythbackend.log file. That might help me help you.
2011-04-06 07:05:33 EST
Subject: watch recording
Comment:
Hey got everything working, but watch recordings will not update at all.
Can you think of any reason why it doesn't? I would get long but don't
know how.
2011-04-06 08:28:23 EST
Subject: Re: watch recording
Comment:
Hi again lowrider,
I've had a problem like that and in my case it
was because I had the wrong MySQL password in the script. I don't know for
sure if you're problem is the same, but if you run this in a
console:
sudo cat /etc/mythtv/mysql.txt
it should tell you
the right username and password that you need to put in the script. If
that's not your problem, post back and I'll see what I can do to figure
this out.
2011-04-07 15:29:50 EST
Subject: watch recording not updating after transcode
Comment:
Hey Defcronyke Webmaster checked my file for to make sure user name and
password where right and they are. Everything is working okay the recoding
the trans-coding its just not updating the watch recoding menu
Just
want to say thanks to you again.
lowrider3121
2011-04-07 21:21:30 EST
Subject: Re: watch recording not updating after transcode
Comment:
Could you please post your MythTV UserJob command, your modified script,
the last 30 lines of your mythbackend.log file captured right after a video finishes
transcoding (in a console, type: sudo tail -30 /var/log/mythtv/mythbackend.log),
what version of MythTV running
on the backend (in a console, type: dpkg -l mythtv-backend), and could you
tell me what version of what OS you're running? I might be able to figure
out what's going wrong if you give me all that info.
2011-04-08 09:17:17 EST
Subject: unit is working
Comment:
Hey Defcronyke Webmaster got the updating to start working used hot key
"D" to delete file that was in there it told me that there was no file
for that recording. But when I deleted it everything started working.
Playing with it today making sure all goes well.
Thanks a lot for
your help.
2011-04-08 16:25:55 EST
Subject: Re: unit is working
Comment:
Hey lowrider,
Glad to hear it's working now. I have no idea why it
got stuck for you, but I'm glad you figured out how to fix it! Please
don't hesitate to come back and ask for help if it turns out that it keeps
getting stuck or it's not working for some other reason. I wish one day
the developers of MythTV would obsolete my script and make some built-in
way of doing what my script does, it would probably be easier to set up if
it was built-in to MythTV.
2011-05-29 14:20:59 EST
Subject: Re: watch recording not updating after transcode
Comment:
I'm having the same problem as lowrider3121. The transcode is working fine
and I end with an .mkv file in the same directory as the original
recording, Family_Guy_-_20101216230000.mkv in my test case this case. But
in the recordings menu it says the file isn't found. I also noticed in
the recordings description the file size hasn't changed, so I'm guessing
the database update isn't working. I don't know anything about SQL so
I'm at a loss for where to go from here, I did double (and triple) check
my db password.
I tried lowrider's suggestion of deleting the
recording entry, I'm not sure what that did for him, but now I just don't
have any entry for that program =p
I'm running mythbuntu 10.04
(lucid)
Mythtv .24 +fixes
I'm using a modified version of your
script to use handbrake as the transcoder, and skipping commercial
cutting.
here's the exact scipt I'm using:
http://pastebin.com/YJ0NxQY7
I'm going to try running it again on
another episode, I commented out "rm update-database.sql" this time so i
can see if there's any problems in there.
I'd appreciate any help
anyone could offer.
Thanks,
FadedSun
2011-05-30 11:57:22 EST
Subject: Found my error
Comment:
I ran another test, this time I didn't delete the update-database.sql
file. the output showed me my error:
UPDATE recorded SET
basename='11_1_KARE-HD_-_915_AM_(Manual_Record)_-_20110530091500.mkv',filesize='123739762',transcoded='1'
WHERE chanid='1111' AND starttime='STARTTIME%';
In the job
command on the mythbackend I missed the % at the beginning of
STARTTIME.
Thanks for the great script, my next transcode worked
fine and the recordings listing updated.
I did notice when playing
back my file in mythtv, fast forwarding and bookmarking didn't seem to
work at all. Fast forwarding didn't seem to do anything, but the program
would end early as if it had. The same with bookmarking, it would still
start at the beginning, but end the program early as if it had started at
the bookmark.
Is this a problem with the way I transcoded my file?
or is it some other problem with replacing the original recorded file in
the recordings database?
Thanks,
FadedSun
2011-05-30 14:43:59 EST
Subject: Re: Found my error
Comment:
Hi FadedSun,
I am running .24 also, but on Debian Squeeze, and I use
the original mencoder version of the script. I can't seem to reproduce
your fast forwarding problem or bookmarking problem with my setup. Both
fast forwarding and bookmarking work as normal for me. I took a look at the
script you're using and I have an idea about a possible fix. Try adding
back this part of the "remove commercials" section (in the correct spot
of course):
# remove commercials
echo "DELETE FROM recordedseek
WHERE chanid='$3' AND starttime='$4';" > update-database.sql
mysql
--user=$DATABASEUSER --password=$DATABASEPASSWORD mythconverg <
update-database.sql
echo "DELETE FROM recordedmarkup WHERE
chanid='$3' AND starttime='$4';" > update-database.sql
mysql
--user=$DATABASEUSER --password=$DATABASEPASSWORD mythconverg <
update-database.sql
#end
This shouldn't actually remove the
commercials, it should just clear out some old seeking information from the
database.
Don't copy-paste it from here,
instead copy the section I posted from
the original downloaded script cause my comments section puts in some nasty
line breaks in bad places. Hopefully everything will work for any new transcodes made with
the fixed script. Good luck and let me know if you have any more trouble.
2011-05-31 19:06:50 EST
Subject: Re: Found my error
Comment:
Clearing the seeking information solved my fast forward and bookmarking
problems. Thank you for all your help!
FadedSun
2011-06-02 18:29:18 EST
Subject: found a bug
Comment:
I ran the script on a recording last night, and the output file name was
"Family_Guy_-_Episode_VI_It's_a_Trap_20110522200000.mkv"
The
database updating portion on the script threw an error on the apostrophe in
the file name:
ERROR 1064 (42000) at line 1: You have an error in
your SQL syntax; check the manual that corresponds to your MySQL server
version for the right syntax to use near
's_a_Trap_20110522200000.mkv',filesize='2008095653',transcoded='1'
WHERE chanid='' at line 1
I haven't tried it yet, so correct me
if i'm wrong, but the following change to the BASENAME should remove
apostrophes from the file name:
BASENAME=`echo "$2" | awk -F/
'{print $NF}' | sed 's/ /_/g' | sed 's/://g' | sed 's/?//g' | sed
s/"'"/""/g`
This is where i got the explanation on how to
remove an apostrophe
http://www.unix.com/unix-dummies-questions-answers/86831-how-replace-special-chars-like-apostrophe.html
Thanks
once again,
FadedSun
2011-06-03 17:37:34 EST
Subject: Re: found a bug
Comment:
Hi FadedSun,
I'm glad the seeking and bookmarking is cleared up for
you. In a future version of the script, I will change the comment for the
portion you added back to say something about what it's actually for, so
it's not confusing anymore. I had never thought about not removing
commercials :)
And thank you for finding and fixing that $BASENAME
bug for me! I really appreciate that you figured it out yourself and gave
me the fix!
2011-07-16 02:26:04 EST
Subject: Possible to use Edit mode on transcoded recording?
Comment:
Hi, this is a really nice script that you put together here. The one
problem I have with it personally from being perfect for me is that after
I've ran it and play the new recording in MythFront, I cannot use the
"Edit mode". It says it doesn't have a seektable immediately after
recordings; I tried using "mythcommflag --rebuild" and that actually got
it to go into Edit mode, but trying to move the cursor backward or forward
freezes the program.
I'm wondering why is this? Because of the
.mkv container or the h.264 codec? Maybe if I used XviD instead it would
be able to edit the MPEG4 stream still, or do you think the container file
would still be the problem?
If so, is there any way to do this
without muxing into .mkv?
Also, is there anyway to make a backup
of the original .mpg recording in case there's some kind of error later in
the program and just deleting it on the condition that all of the encoding
programs exit successfully? I'm not really sure why the .mpg recording is
turning into .mpg.tmp
2011-07-18 12:27:44 EST
Subject: Re: Possible to use Edit mode on transcoded recording?
Comment:
Hi Kenneth,
To be honest, I have never used "edit mode" and
actually don't know what that is, but I have found that recordings
converted to H.264 in the .mkv container don't work with the mythtv seek
table. The script erases the seek table to prevent that very bug you
describe, where it freezes, or refuses to seek at all during
playback.
So it looks like you'll have to try another format if you
need to keep the seek table and also want to watch the video inside
Mythfrontend. You could avoid my script altogether, and just use the
built-in transcoder, as it will produce files that work with a seek table.
Another option would be to examine the transcoding commands in the script
(oggenc, mencoder, mkvmerge), and change them so that they transcode to a
different format/container.
So far from the testing I've done, it
seems only the h.264/mkv files don't work with a seektable. Good luck, and
feel free to come back here if you need any more help.
2011-07-24 07:47:24 EST
Subject: none
Comment:
I sought your script out as an alternative to the built-in transcoder
because I didn't feel it offered very good results. It needs far too much
file size for video quality.
After some testing though I did find
that the matroska container was the biggest impediment. The "edit mode"
I speak of is just a mode that enables the user to mark a cutlist in order
to edit out commercial breaks manually as the detection is pretty
unreliable. In any case, I found that after encoding to just a raw avi
file format either XviD or h.264 will work and one can build a cut list and
seek and everything of that nature, but for some reason once your commands
for removing commercials has run on an .mpg mencoder can no longer open the
file.
2011-10-02 20:53:10 EST
Subject: Won't process Video files
Comment:
Again, Thank you so much for this script!! On my myth box I seem to be
processing the audio (I see the files show up in the temp directory then go
away). I have been fighting to get this to work for 2 weeks now. I have
been running 5 minute recordings to test it. It deletes the original
recording but does not create anything. I found an error in the back end
log for the mencoder. I'm not sure how to adjust the script without
messing it all up.
My Log:
VDec: vo config request - 480 x
480 (preferred colorspace: Planar YV12)
VDec: using Planar I420 as
output csp (no 1)
Movie-Aspect is 1.33:1 - prescaling to correct movie
aspect.
[CROP] Bad position/width/height - cropped area outside of the
original!
FATAL: Cannot initialize video driver.MEncoder
SVN-r1.0~rc3+svn20090426-4.4.3 (C) 2000-2009 MPlayer Team
success:
format: 0 data: 0x0 - 0xaf1f004
MPEG-PS file format detected.
VIDEO:
MPEG2 480x480 (aspect 2) 29.970 fps 6000.0 kbps (750.0 kbyte/s)
[V]
filefmt:2 fourcc:0x10000002 size:480x480 fps:29.970
ftime:=0.0334
Opening video filter: [expand osd=1]
Expand: -1 x -1,
-1 ; -1, osd: 1, aspect: 0.000000, round: 1
Opening video filter:
[harddup]
Opening video filter: [scale w=512 h=384]
Opening video
filter: [yadif]
Opening video filter: [crop w=704 h=528 x=12
y=0]
Crop: 704 x 528, 12 ;
0
==========================================================================
Opening
video decoder: [mpegpes] MPEG 1/2 Video passthrough
VDec: vo config
request - 480 x 480 (preferred colorspace: Mpeg PES)
Could not find
matching colorspace - retrying with -vf scale...
Opening video filter:
[scale]
The selected video_out device is incompatible with this
codec.
Try appending the scale filter to your filter list,
e.g. -vf
spp,scale instead of -vf spp.
VDecoder init failed :(
Opening video
decoder: [ffmpeg] FFmpeg's libavcodec codec family
Unsupported
PixelFormat -1
Selected video codec: [ffmpeg2] vfm: ffmpeg (FFmpeg
MPEG-2)
Exiting...
MEncoder SVN-r1.0~rc3+svn20090426-4.4.3 (C)
2000-2009 MPlayer Team
success: format: 0 data: 0x0 -
0xaf1f004
MPEG-PS file format detected.
VIDEO: MPEG2 480x480
(aspect 2) 29.970 fps 6000.0 kbps (750.0 kbyte/s)
[V] filefmt:2
fourcc:0x10000002 size:480x480 fps:29.970 ftime:=0.0334
Opening video
filter: [expand osd=1]
Expand: -1 x -1, -1 ; -1, osd: 1, aspect:
0.000000, round: 1
Opening video filter: [harddup]
Opening video
filter: [scale w=512 h=384]
Opening video filter: [yadif]
Opening
video filter: [crop w=704 h=528 x=12 y=0]
Crop: 704 x 528, 12 ;
0
Modified part of script:
# a temporary working directory
(must be writable by mythtv user)
TEMPDIR="/shows/myth_temp"
#
MySQL database login information (for mythconverg
database)
DATABASEUSER="mythtv"
DATABASEPASSWORD="LAsOZ21H"
#
MythTV Install Prefix (make sure this matches with the directory where
MythTV is installed)
INSTALLPREFIX="/usr/bin"
My start
script is:
/shows2/mythtv-transcode-h264.sh "%DIR%/%FILE%"
"%DIR%/%TITLE% - %PROGSTART%.mkv" "%CHANID%" "%STARTTIME%"
My
Mythbackend ver:
Mythtv-backend 0.23.0+fixes24
Thank you so
much for your help.
Pat
2011-10-03 10:47:51 EST
Subject: Single Recording Great, multiple recordings not so great
Comment:
I have been using your script with recordings from my HDHR and it has
worked great (after I removed all the cropping and scaling)
However
if more than one job of running this script is happening the end files get
pushed together.
Example. I recorded a football game (long
recording) and after it was done recording and this user job was running,
HDHR was recording the Simpsons, and later another show. The next morning
the video for the football game could not be found, the Simpson's video
played, but had the audio of the football game.
Any tips of where I
can look for more clues?
Thanks again for putting this together.
2011-10-05 14:15:59 EST
Subject: Re: Won't process Video files
Comment:
Hi Emt35,
Try removing the cropping and scaling from your mencoder
chain in the script.
Change both lines that look like this:
#
first video pass
mencoder -nosound -vf
crop=704:528:12:0,yadif,scale=512:384,harddup "$1.tmp" -o /dev/null -ovc
x264 -x264encopts pass=1:bitrate=1000:threads=2
# second video
pass
mencoder -nosound -vf crop=704:528:12:0,yadif,scale=512:384,harddup
"$1.tmp" -o $AVIFILE -ovc x264 -x264encopts
pass=2:bitrate=1000:threads=2
Remove the crop and scale options
so that they look like this:
# first video pass
mencoder -nosound -vf
yadif,harddup "$1.tmp" -o /dev/null -ovc x264 -x264encopts
pass=1:bitrate=1000:threads=2
# second video pass
mencoder
-nosound -vf yadif,harddup "$1.tmp" -o $AVIFILE -ovc x264 -x264encopts
pass=2:bitrate=1000:threads=2
From your log it looks like the crop
option was failing, so without the cropping and scaling I would think it
would probably get past that problem. Let me know if you still have trouble
after trying that.
It looks like your capture device is capturing at
480x480 which is pretty low-res. This could be why the cropping was
failing. If you know that you should be capturing at that low resolution
that's fine, but I think lots of capture cards can capture at least
720x480, so you might want to look into your capture card settings and see
about capturing at a higher resolution.
2011-10-05 14:30:14 EST
Subject: Re: Single Recording Great, multiple recordings not so great
Comment:
Hi Ronoc,
I'm sorry the video for your game got deleted, and that
the audio took over your Simpsons episode. This probably happened because
the script uses a static working directory (the same dir for any job that
might be running). I have only tested it running on one recording at a
time, so I never ran into that problem, but it's a big problem, and I will
do my best to fix it for the next release.
In the meantime, if you
want to hack at it yourself, I would suggest creating a new subdirectory in
the $TEMPDIR for each job instance (a folder whose name is based on the
show title), and make sure each job instance only uses its own unique
subfolder for all the temporary work.
Thanks for the bug report and
let me know if you figure it out. Otherwise, check back here in the next
couple months for an updated version of the script with that problem
(hopefully) fixed.
2011-10-06 13:47:56 EST
Subject: leave flagged commercials in recordings
Comment:
Hey friend, thanks so much for sharing this script. If I want to leave the
already flagged commercials in the recordings, it is safe to merely comment
out the lines in "remove commercials" section of the script? The wife
does not trust mythtv to always correctly flag commercials and nothing more
(sometimes too much is flagged such as upcoming scenes in next week's
episode.)
(Running 32-bit Mythbuntu 10.04 on a AMD Althon 64 dual
core 2.5 Ghz, Hauppauge HVR-2250, GT430, 2GB RAM, 1 TB WD Blue
HD.)
Thanks again for sharing this.
Kind Regards.
2011-10-06 18:19:52 EST
Subject: Re: leave flagged commercials in recordings
Comment:
Hi stlouisubntu,
You should be able to keep the commercials in the
transcoded file by commenting out the section under # remove commercials,
but I wouldn't go so far as to say it's safe (I haven't tested it). Try
it if you want, and if you try it, let me know if it works.
2011-10-08 11:08:28 EST
Subject: Won't process Video files
Comment:
Thank you that worked! When I set this DVR up I had a small hard drive
and set the resolution low to save space, I never went back when I added
more hard drives, I'll have to look at that. My test worked a 5 minute
recording took 25 minutes to process (I have an old HP Pentium 4). Thanks
for your help!!
2011-10-09 22:01:08 EST
Subject: Recording Unavailable "Title" The file for this recording can not be found
Comment:
Thanks for the advice. I did attempt one recording (no changes to the
remove commercials section yet) as a test and it appears in my list of
recordings but with a X next to it and when I try to launch it to play, I
get the error message Recording Unavailable "Title" The file for this
recording can not be found. Any advice on what I might change to fix
this?
Thanks.
2011-10-10 12:06:12 EST
Subject: Re: Recording Unavailable "Title" The file for this recording can not be found
Comment:
Hi stlouisubntu,
That is strange behaviour. I can only think of one
thing that might cause that. Take a second look at your User Job#1 command.
It should look like this:
/path/to/script/mythtv-transcode-h264.sh
"%DIR%/%FILE%" "%DIR%/%TITLE% - %PROGSTART%.mkv" "%CHANID%"
"%STARTTIME%"
Make sure all the % are there. Did you miss one next
to TITLE?
2011-10-27 06:31:39 EST
Subject: Feedback and Thanks!
Comment:
Just wanted to say thanks for your script.
I've updated it with the
following encoder options, which seem to work for me, providing even better
quality - based on a good number of HD channels I get now. The key goal
for me however, was not small size, but better performance on playback,
utilising h264 hardware decoders.
mencoder -nosound -noskip -vf
yadif,harddup "$1.tmp" -o /dev/null -ovc x264 -x264encopts
pass=1:turbo:threads=2
mencoder -nosound -noskip -vf yadif,harddup
"$1.tmp" -o $AVIFILE -ovc x264 -x264encopts
pass=2:bitrate=1500:threads=2
mkvmerge -o "$DIRNAME/$BASENAME" -A
$AVIFILE -y 0:0 $AUDIOFILE
I'm also not encoding to ogg, but
leaving the PCM stream intact, so I can get the full surround
sound.
What I would like to know - if possible, is can I set mythtv,
so that EVERY recording gets this transcoding, by default.
Currently,
just commercial-skipping is on by default and I need to either set the
transcode in the recording profile, or transcode afterwards.
2011-10-27 15:56:30 EST
Subject: Re: Feedback and Thanks!
Comment:
Hi Stevellion,
Thanks for sharing your mencoder chain.
As for
setting the transcoding on by default, simply open Mythfrontend,
then
navigate to: TV -> TV Settings -> General -> [press "next"
twice]...
at the bottom where it says "Default Job Queue Settings for
New Scheduled Recordings", check the box next to any userjobs that you
would like to be on by default for all your scheduled recordings.
2011-12-08 12:41:52 EST
Subject: Thank You
Comment:
I wanted to say thanks for the script. I will say that the hvr-2200
recordings were failing the transcode due to their frame size being less
than the mencoder crop size. I removed the crop option and all is good now.
Can you see any reason as to why I would need that?
Thanks again!
2011-12-08 12:41:10 EST
Subject: Re: Thank You
Comment:
Hey unixfreak,
I think I had the crop in there to get rid of some
garbage on the edge of the screen that my PVR-150 was giving me. You
probably don't need it.
2012-01-14 21:34:06 EST
Subject: Permission Errors
Comment:
Hi.The mythbackend log file is giving me a long list of permission errors.
I've double checked that I followed all your instructions but I can't
figure out what I've done wrong.
2012-01-24 19:28:22 EST
Subject: Couple suggestions
Comment:
If mythbackend runs two jobs at the same time your mysql commands could get
crossed and confused. I suggest doing something like the following which
will use the process-id to name the command tmp file. You will need to
change them all, just not the ones in my example.
--
MYPID=$$
echo
"DELETE FROM recordedmarkup WHERE chanid='$3' AND starttime='$4';" >
update-database-$MYPID.sql
mysql --user=$DATABASEUSER
--password=$DATABASEPASSWORD mythconverg < update-database-$MYPID.sql
rm
update-database-$MYPID.sql
2012-01-24 19:32:38 EST
Subject: Nice your process
Comment:
# Below commands will result in these tasks using idle resources in
attempts to keep other tasks responsive
MYPID=$$
renice 19 $MYPID
# Make this process low priority with the CPU scheduler
ionice -c
3 -p $MYPID # Make this process low priority with the disk IO
scheduler
2012-01-24 19:37:34 EST
Subject: User definable value for threads
Comment:
The mencoder command syntax in your examples defines threads=2 which is
optimal for a 2-core system but probably not ideal for 1 of 4 core
system.
ex) mencoder -nosound -vf
crop=704:528:12:0,yadif,scale=512:384,harddup "$1.tmp" -o /dev/null -ovc
x264 -x264encopts pass=1:bitrate=1000:threads=2
I suggest setting
this to threads=auto which will detect how many cores you have
ex)
mencoder -nosound -vf crop=704:528:12:0,yadif,scale=512:384,harddup
"$1.tmp" -o /dev/null -ovc x264 -x264encopts
pass=1:bitrate=1000:threads=auto
You could even create a user
definable variable at the top and allow users to define this setting in one
place
ex)
$NUMTHREADS=auto
mencoder -nosound -vf
crop=704:528:12:0,yadif,scale=512:384,harddup "$1.tmp" -o /dev/null -ovc
x264 -x264encopts pass=1:bitrate=1000:threads=$NUMTHREADS
2012-02-08 18:48:14 EST
Subject: Re: Couple suggestions
Comment:
Hey ScottZ,
Thank you for all your suggestions! I will definitely
implement all of them for the next version of the script. Great
suggestions!
2012-02-09 12:39:47 EST
Subject: Script updated!
Comment:
v0.8 now works for multiple simultaneous userjobs, with increased niceness,
and defaults to using all processor cores.
Thanks to Ronoc who
pointed out the multiple jobs bug, and a huge thanks to ScottZ for offering
examples on how to improve the script!