This is a guide that basically combines protolium's very helpful ffmpeg cheatsheet with the spleeter library.

Here's a tweet thread that shows a video snippet, with separate bass, vocals, and drums track:

Installing stuff

Isolating instruments from music is now possible using AI, and Splitter is based on Deezer's open source research project Spleeter to accomplish this. Currently a few different models are available; 2 stem and 5 stem models. I've succeeded to build standalone Mac Application for separation(5stems). No needs to interact with Terminal(command line). Only you have to is to download my 'MySpleeter' app from below because this app includes python, spleeter, ffmpeg, tensowflow and dependencies. (changed at 2020/09/04).

1. 音楽データからボーカル・ドラム・ベースの音を個別に抽出できる「spleeter」 ざっくりな機能は以下を見てもらえればわかるかと。 mac にインストールして使えるようにしてみます。 ※実施日は 2021/04/08 環境 インストール Miniconda インストール （※Python環境構築済みの場合は不要） spleeter.
2. A couple of weeks back, Deezer released its new AI-based tool to separate vocals from songs. However, setting it up and running it requires installing various packages that can be time-consuming. To get around this, a developer has created a website based on Deezer Spleeter that lets you seamlessly split songs.

ffmpeg

(this guide was tested on ffmpeg 4.1.4)

Lots of ways to do this. If you're on a Mac, I suggest using Homebrew:

spleeter

Repo: https://github.com/deezer/spleeter

HN discussion: https://news.ycombinator.com/item?id=21431071

Install (and isolate) using conda:

You can also do pip install spleeter but you might run into more than a few library conflicts with your existing Python environment...

Extracting the audio from video

(From this helpful gist: https://gist.github.com/protrolium/e0dbd4bb0f1a396fcb55)

Given a video.mp4 file, here's how to use ffmpeg to extract the audio into a separate file, audio.wav:

Splitting the audio into 5 tracks with spleeter

(From spleeter's Quick Start info: https://github.com/deezer/spleeter#quick-start)

The following command invokes spleeter to split audio.wav into 5 separate tracks, into a subdirectory called myoutput/audio

The generated files:

Creating a video file with one of the extracted tracks

Basically, create a silent version of the video, and then merge it with one of the given track extracts.

Strip audio from video

This creates a copy of the video, sans audio track, named silent-video.mp4 from video.mp4

Merge silent video with audio track

Here's an example that creates drums-video.mp4: the video, with just the drums track:

Make an excerpt of a media file (optional)

(StackOverflow: https://superuser.com/questions/377343/cut-part-from-video-file-from-start-position-to-end-position-with-ffmpeg)

If you want to snippet any media file, here's an example using ffmpeg that cuts a 12-second snippet, beginning at the 30th second:

Spleeter Audio

Spleeter is the Deezer source separation library with pretrained models written in Python and uses Tensorflow. It makes it easy to train source separation model (assuming you have a dataset of isolated sources), and provides already trained state of the art model for performing various flavour of separation:

• Vocals (singing voice) / accompaniment separation (2 stems)
• Vocals / drums / bass / other separation (4 stems)
• Vocals / drums / bass / piano / other separation (5 stems)

2 stems and 4 stems models have state of the art performances on the musdb dataset. Spleeter is also very fast as it can perform separation of audio files to 4 stems 100x faster than real-time when run on a GPU.

Spleeter Download

We designed Spleeter so you can use it straight from command line as well as directly in your own development pipeline as a Python library. It can be installed with Conda, with pip or be used with Docker.