VidCoder Settings Tutorial - High Quality H.264 (x264) MKV Video Encoding

Discussion in 'Media Guides' started by Hoffman, May 14, 2012.

  1. Hoffman

    Hoffman Addict

    Joined:
    Jun 1, 2007
    Messages:
    288
    Likes Received:
    369
    How to encode high quality x264 / MKV/ MP4 video with VidCoder, powered by handbrake.

    This tutorial aims to teach you how to produce high quality x264 encodes of your Blu-rays, DVD's and any other video you might have on your hard disk.

    First of all, what is VidCoder?

    VidCoder is a free, open source Blu-ray & DVD ripping / video transcoding application for Windows. It uses HandBrake as its encoding engine.

    Why should i use VidCoder?
    • Multi-threaded
    • MP4, MKV containers
    • H.264 encoding with x264, the world's best video encoder
    • Completely integrated encoding pipeline: everything is in one process and no huge intermediate temporary files
    • MPEG-4, MPEG-2, Theora video
    • AAC, MP3, Vorbis, AC3, FLAC audio encoding and AAC/AC3/MP3/DTS/DTS-HD passthrough
    • Target bitrate, size or quality for video
    • 2-pass encoding
    • Decomb, detelecine, deinterlace filters
    • Batch encoding
    • Instant source previews
    • Creates small encoded preview clips
    • Pause, resume encoding

    Most of all this is all FREE, legally.

    Where can i download VidCoder?

    Download VidCoder 1.3.2 32bit
    Download VidCoder 1.3.2 64bit

    Now you are sold lets get started!

    To Begin:

    Choose your movie to encode on the main window and choose the [High Profile] preset to base your settings off of. It is important to first pick the [High Profile] preset and then go to the settings window to make your changes.

    1-vidcoder-mainmenu.png


    Picture Tab:

    - Keep the width the same as the original width (usually 720 for standard dvds) and make sure
    "Keep Aspect Ratio" is checked.
    - Anamorphic: Choose [Loose]. Don't worry if the display resolution says something different
    than the storage resolution.
    - Cropping: Choose [Automatic]

    2-vidcoder-settings-picture-tab.png


    Video Filters Tab:

    - Detelicine: Choose [Default], this setting is usually safe to keep at default.
    - Decomb: Choose [Default], this setting is a smart deinterlacer. It will make sure to deinterlace
    when needed...but if and only if when needed. It is therefore safe to leave on even if your source
    is not interlaced.
    - Deinterlace: [Off] No need for this since Decomb is smarter and will deinterlace if needed.
    - Denoise: usually [Off]. However, if you have a very grainy, noisy source video, choose [Weak]
    to smooth out some of the noise/grain. Choose [medium] only in very rare cases. Denoise
    smooths out the picture, but it also eats away at detail. Therefore, I rarely use this setting. But if I
    have a very noisy video and/or want to get the file size down a bit (denoise reduces detail, and
    therefore also reduces the need for higher bitrate), I usually choose [weak].
    - Deblock: [Off]

    3-vidcoder-settings-video-filters-tab.png


    Video Tab:

    - Video Codec: [H.264(x264)]
    - Framerate (FPS): Depends. Usually [23.976]. If you know you have a constant framerate,
    leave this at [Same as Source]. If you have a source with a variable framerate, you may have to
    try encoding at both 23.976 and 29.97 to see what looks better. In most of my cases, 23.976 has
    usually been the choice for sources with variable framerates. Note, Vidcoder may tell you (in the
    blue box) that the input framerate is 29.97 when it may actually be variable. If you know or find
    out that it is variable, and Vidcoder says input is 29.97, 23.976 may actually look better. So
    bottom line, if you have a variable framerate source, choosing same as source will probably break
    compatibility with various media streamers. Therefore you are forced to choose between 23.976
    or 29.97. Although 9 times out of 10, I have gone with 23.976...you should still test encoding,
    trying each one.
    - Target Size: Depends. More on how to choose this later.
    - 2-Pass Encoding: turned [on] if using Target Size
    - Turbo First Pass: turned [off]

    4-vidcoder-settings-video-tab.png


    Audio Tab:

    - Choose your audio stream (usually first one...other streams may include commentary track,
    other languages, etc.
    - Choose [AC3/DTS Pass-through].
    - AC3/DTS Pass-through "passes through" the audio. Meaning, the audio in your encoded file will
    be the exact same quality audio track that's included with your source file. Anything besides
    Pass-through will be lossy, meaning you will lose some quality. The only reason to "downmix" to
    another codec such as AAC is if your target device doesn't support AC3.

    5-vidcoder-settings-audio-tab.png


    Advanced Tab:

    Encoding:
    Reference Frames: 8
    Maximum B-Frames: 8
    CABAC: Checked (on)
    8x8 Transform: Checked (on)
    Weighted B-Frames: Checked (on)
    Pyramidal B-Frames: [Normal (default)]

    Psychovisual:
    No DCT-Decimate: unchecked (off)

    Analysis:
    Adaptive B-Frames: [Optimal]
    Adaptive Direct Mode: [Automatic]
    Motion Estimation Method: [Uneven Multi-Hexagon]
    Subpixel Motion Estimation: [10]
    Motion Estimation Range: [32]
    Adaptive Quantization Strength: left untouched (default)
    Psychovisual Rate Distortion: left untouched (default)
    Psychovisual Trellis: [0.10] (two notches up from the default which is 0)
    Deblocking: [0, 0] (default)
    Partition Type: [All]
    Trellis: [Always]

    Manual Adjustment to Options String:
    On the Advanced tab, you will see a box for "Options String" at the bottom. Manually change rclookahead
    from 50 to 100. There is no way to change this value from the gui, however, it can be
    manually changed in the options string box.

    Options String:
    When you are done choosing all of your options, the Options String should look like this:
    [rc-lookahead=100:ref=8:bframes=8:badapt=
    2:Direct=auto:me=umh:Subme=10:merange=32:analyse=all:trellis=2:Psy-rd=1.0,0.10]


    Preset:
    - Click on "save as..." at the top of the settings window to save your settings for future use.

    6-vidcoder-settings-advanced-tab.png

    Final Notes:
    These settings require a lot of juice from your computer. I have an Intel i7 (quad core / hyperthreading / turbo boost) with 8gb of ddr3 ram. It really helps to have a powerful processor for encoding. With my system and these settings, I get an average encoding speed of about 12-15fps. A two-pass encode takes about 3 hours. It's lengthy. But to me it's worth it.

    These settings create very high quality videos with an acceptable file size (higher settings = better picture and better compression). So my computer basically always has Vidcoder running. If these settings are just too much for your computer to handle, and you can't get a new computer : ), the first setting you may want to adjust down is the Motion Estimation Range. 32 is kind of overkill. Higher values are good for high action videos. But you can get away alright with the default which is 16. You will see an increase in speed if you lower this value.

    Then I would next maybe drop Subpixel Motion Estimation from 10 to 9. If you still need to boost your speed, you could try lowering your Reference Frames and Maximum B-Frames from 8 to 6 or even lower. Just remember, you may then have to compensate for less compression and up your target size. You could also turn on [Turbo First-Pass]. You'll see almost no difference from regular 2-pass.


    How to Choose your Target Size:

    So how do we know what target size to use? After all, different video sources require different bitrates (thus different sizes) to achieve the same quality. An old grainy video may need a higher bitrate and require a larger size. Every video is a little different and we shouldn't take a "one-sizefits- all" approach. The standard of old was: 350mb for a 1 hour (actually around 45 min without commercials) show. And forget 5.1 audio...that took up too much space. It was 2-channel mp3
    128 bitrate. I don't know how this precedent got started, but I could strangle whoever came up with this. 350mb is almost never enough to create a good looking video...even with the high compressing x264 codec. Perhaps it was because then you could fit two (blocky looking) episodes on one cd (700mb). Well, we no longer rely on cd's. We have hard drives and flash drives. Hard drive space is relatively cheap, so let's worry about quality over size. Especially if you plan on streaming your files to a TV.

    x264 comes with a setting called "Constant Quality", often referred to as CRF. This is a great feature. Basically, you set it at a value, and then let the encoder figure out what bitrate/size is needed to attain that quality. There are a couple benefits. One is that you don't have to do 2-pass which takes longer. CRF does everything in 1 pass. Also, you don't have to guess the bitrate or target size. The encoder will figure that out and will smartly give you the desired quality every time. It is an amazing feature, and I have no idea how it works. But that's besides the point. I usually use CRF for my movies. I generally don't care how big the size is of my movie files. However I do care about the quality. So I pick my quality setting and let the encoder figure everything else out. However, for TV. shows, I generally like to predict my target size so I can be consistent with all episodes of every season across the board. And after all, generally t.v. episodes within a series require about the same bitrate to maintain the same quality, since the original sources are very similar.

    Here's how I go about determining what target size to use:

    With each new series, I begin with some lengthy experimenting. I try to take a random sample...a few episodes from different DVDs and from different seasons. I begin by encoding each episode 3 different times with CRF levels of 18.5, 19, and 19.5. These numbers should all give a pretty good acceptable quality. I always do 18.5 with my movies, but 19 is a pretty good rule for standard DVDs. (For Bluray, you can get away with higher CRF values like 22 or 23). Then I see what kind of sizes I get. You will see that they vary, but usually by not too much. Some episodes will need a lesser bitrate to attain the very same quality as an episode that may require 50-75 more mb's.

    Examine the sizes of the resulting files and pick an average. You will get a feel for how large you should go. I usually find that it hovers any where from 450 - 600mb...sometimes more, especially if you include multiple audio tracks. And remember, a lot of the compression, and thus size, is based on your x264 settings. For example, say you use a Subpixel Motion Estimation of 8 for one encode and 10 for another...on the same episode/file, but you choose the same CRF value of 18.5 for both. The one where you set the SME at 8 will be a larger output file than the one you set at 10. Because the encoder was told to maintain the same quality. So at 10, it could keep the same quality with a lower bitrate because it had more compression. But an SME of 8 required a higher bitrate to maintain the same quality. I hope this makes sense. Basically. the better the x264 settings, the smaller file size you will need to maintain the same quality.

    I think that about covers everything. If you want to add subtitles, I generally do this after the fact. I use Vsrip to create the individual episodes .idx/.sub files and then use Subrip to convert them to .srt. Then I mux them into the mkv file with mkvmerge gui.
     

    Attached Files:

    Dark Scyth, Rick and Fejinwales like this.
  2. Dark Scyth

    Dark Scyth Moderator

    Joined:
    Mar 3, 2009
    Messages:
    2,594
    Likes Received:
    459
    Any idea on how this encodes the MKV? Is it 8bit or 10bit?
     
  3. Fejinwales

    Fejinwales Member

    Joined:
    Feb 5, 2013
    Messages:
    10
    Likes Received:
    0
    I just want to thank you for this post, it made a massive difference to my films and although it takes a long time, even on my PC, it is definitely worth the wait.
     
  4. seanpr92

    seanpr92 Godlike

    Joined:
    Oct 12, 2010
    Messages:
    1,159
    Likes Received:
    195
    Location:
    Nottingham, United Kingdom
    i made some optimal settings for youtube for MKVs
     

    Attached Files:

  5. carlmart

    carlmart Member

    Joined:
    Mar 9, 2013
    Messages:
    11
    Likes Received:
    0
    Hoffman,

    Thank you very much for this tutorial.

    My hardware is already high-profile ready, even if at the moment I'm converting still within XP. As soon as I solve some streaming issues I have with my Windows 7 network, I should settle there.

    Vidcoder seems like an excellent tool for what I usually need to do, which is lowering bitrate on 1080p video files so they can fit on a single 8Gb DVD-DL.
     
  6. carlmart

    carlmart Member

    Joined:
    Mar 9, 2013
    Messages:
    11
    Likes Received:
    0
    OK. Just finished my first conversion using these encode setups.

    I was able to do it within Windows 7 64, using its full i7 and 8Gb capacility. Even so it took 9 hours to accomplish the conversion.

    What I was doing was converting an 1080p mkv file, 100 minutes long, onto another 1080p file with an 8Gb size.

    Is that about right?

    What I could observe, during conversion, is that Vidcoder was not using the full CPU or all memory to process it. Memory used was just about 3Gb. Is that correct?
     
  7. InsaneNutter

    InsaneNutter Resident Nutter Staff Member

    Joined:
    Jun 1, 2007
    Messages:
    12,254
    Likes Received:
    3,724
    Location:
    Yorkshire, England
    Memory used sounds normal, however it should be able to max out all your CPU cores.

    What percentage of your CPU was been used?

    As for the time it does take a lot longer when you specify the size, especially with the settings here. However you will have a fantastic encode at the end of it.
     
  8. carlmart

    carlmart Member

    Joined:
    Mar 9, 2013
    Messages:
    11
    Likes Received:
    0
    Yes, I was wrong. Vidcoder is using all the 8 multithread cores.

    About specifying size, I was working on the idea of putting as much possible bitrate as you can get for the max size I am basing my conversions on, as long as it does not exceed the original bitrate, of course.

    Because of specific reasons of my mkv player, which sometimes won't play DTS audio tracks, what I always do is convert DTS to Dolby 5.1, @ 640 kbps. I think this is a very minor sacrifice to get more "space" to increase bitrate or to get at least to 10000Mbps, which I think is a good figure for that spec. So now I am not processing any sound altogether with Vidcoder, leaving the space for the future Dolby tracks. Am I right in doing so?

    What I do with the audio, after extracting it, is use Avisynth to convert it to AC3. This has been working quite well for years. Then I add it on the final merge, where I also put the subtitles.

    One thing that happened on my last conversion was weird too: I specified a final size, and when conversion was done Vidcoder had changed that size to a much smaller one. My size target was 7400 and it ended up with 4533. Any reason why that might happen?
     
  9. carlmart

    carlmart Member

    Joined:
    Mar 9, 2013
    Messages:
    11
    Likes Received:
    0
    Processing time is a bit long. Last one took 18:13 hours to process, with nothing else running at the same time.

    File was 11.33Gb and 2:57 hours long. But now I saw it also converted the resolution from 720p to 1080p.

    How do I set things up so conversion always keeps the original resolution?
     
  10. carlmart

    carlmart Member

    Joined:
    Mar 9, 2013
    Messages:
    11
    Likes Received:
    0
    In spite of playing fine, all videos I have been converting using this setup got me a warning on my LG BD390 player: this movie may not be playable properly.

    This is the player I use to play my mkv files with, and it's the only way I view my videos with. It's the most practical I have experienced, and I did try using my laptop as media center.

    What I wonder is what item might fire this warning I mention above. It's not something in the original file, it's in the processed file. I don't care trying different setups by splitting the video in 1 minute pieces, but at least I would like to have some guidance.

    What values do you think I might adjust?
     
  11. InsaneNutter

    InsaneNutter Resident Nutter Staff Member

    Joined:
    Jun 1, 2007
    Messages:
    12,254
    Likes Received:
    3,724
    Location:
    Yorkshire, England
    Firstly despite the warning does it play ok?

    What happens when you take the default high profile setting, specify your resolution and just let Vidcoder encode to a constant quality?

    Thats where i would start, then change settings until you find something that your player doesn't like.
     
  12. carlmart

    carlmart Member

    Joined:
    Mar 9, 2013
    Messages:
    11
    Likes Received:
    0
    Yes, despite the warning it plays OK.

    About the settings, another option would be to compare the original to the encoded file settings, extracted with Mediainfo, and work on those that are different, if possible.

    What do you think?

    I just asked you first because you might know which variables may be critical on some bluray players.
     
  13. carlmart

    carlmart Member

    Joined:
    Mar 9, 2013
    Messages:
    11
    Likes Received:
    0
    OK. What I did now is, using Mediainfo advanced data, compare encoding settings of the original and the one I'm outputting using the settings you propose.

    Then I deleted the values that were the same and left just those that were different.

    What I wonder is what should I do about them. Should I change those that I can, as I'm not sure changing them will improve anything, or would it? Some of them are not listed as options on VC's setup, so what should I do about them?

    A few things worry me:

    1) As I'm changing bitrate by establishing the file size, is there anything I should compensate with?

    2) How to sped up conversion time without affecting quality.

    3) Several places I am looking at work with PC viewing as their reference, where monitor sizes are not that large. My reference are plasma monitors, larger than 50", even if now I have a 42".
     

    Attached Files:

  14. carlmart

    carlmart Member

    Joined:
    Mar 9, 2013
    Messages:
    11
    Likes Received:
    0
    Some days ago I tried something, which is is import Mediafile XML file, but it didn't work, got me an error.

    The weird thing is that I'm also getting an error importing preset files generated by Vidcoder itself.

    The idea is to use most of the encoding presets used on the original file.
     
  15. carlmart

    carlmart Member

    Joined:
    Mar 9, 2013
    Messages:
    11
    Likes Received:
    0
    Hi,

    Well, I'm back on the saddle again, doing some more VC conversions.

    As apparently I misplaced my old High Profile setup, I went all over it again following your instructions at the beginning. And I am processing a video as we speak.

    What I'm finding a bit long are the ETA for this film I'm processing, which is 15H 32min. which usually ends up longer than that. If I recall, my previous conversions took about 8 hours to accomplish, and the hardware is the same.

    So maybe I should change some setting to get things more manageable without sacrificing quality.

    What do you suggest I should do?


    Carlos
     
  16. InsaneNutter

    InsaneNutter Resident Nutter Staff Member

    Joined:
    Jun 1, 2007
    Messages:
    12,254
    Likes Received:
    3,724
    Location:
    Yorkshire, England
    I've attached my settings I used for some recent 1080p Blu-ray encodes. These were used on some 1h 30min (ish) movies, these settings probably took about 6-8 hours to encode the movie on a Core i5 2500 @ 3.3 ghz.

    You can probably speed things up by enabling "turbo first pass"

    turbo-first-pass.jpg

    Also note i use a target file size, i generally follow the sizes "the scene" use for movies:

    1h 30ish movie = 8gb file size
    2h to 2h 20ish movie = 10gb file size

    I have never had any quality problems doing so.

    You could also select constant quality and enter 10 or 12 and see what quality of encode you get them, encoding to a constant quality is supposed to be a lot quicker, however i prefer to know the file size i'm going to end up with.

    Hope thats some help anyway, i'm no expert, i've just found the settings i've attached work well for me.
     

    Attached Files:

    Rick likes this.
  17. carlmart

    carlmart Member

    Joined:
    Mar 9, 2013
    Messages:
    11
    Likes Received:
    0
    Well, things got much better in time when I set for constant frame rate and added Turbo First pass.

    I need the total size always to be 8Gb, even if that may limit the bitrate.
     

Share This Page