Viba IT FreeDB to MySQL parser (VFM) is a small program to parse the FreeDB and insert the data into a MySQL database. The parser is written in Java and licensed under the GPL.

This parser will at this stage in development only parse the fields for disc length, DISCID, DTITLE, DYEAR, DGENRE and the TTITLE fields. Depending on what your purpose is with the data this may be enough.

Requirements

This first version is rather memory consuming and you should make sure to have atleast 170 Mb free RAM before you parse the "misc" or "rock" directories. It is the implementation of the directory listing that uses some memory. A better solution is prepared for but not implemented.

Except the hardware requirements you should have Java version 1.4.2 or higher and the MySQL Connector/J version 3.1.12 or higher. The parser have currently only been tested on Ubuntu Linux running AMD64. If you successfully run the parser on another system then please contact us.

Installation

We assume that you already have downloaded and unpacked the FreeDB into a directory on your harddrive. For help with this, see the FreeDB FAQ.

For the parser to be able to connect to the MySQL server you'll have to have the JDBC driver for MySQL in the CLASSPATH of Java. If you're using a Linux system and bash, do like this.

If you don't have the MySQL Connector/J installed, download it and unpack the archive.

$ cd /path/to/downloaded/file/
$ tar zxf mysql-connector-java-3.1.12.tar.gz
$ cd mysql-connector-java-3.1.12
$ export CLASSPATH=$CLASSPATH:`pwd`/mysql-connector-java-3.1.12-bin.jar

Now download the Viba IT FreeDB to MySQL parser (11.4 kB) and do like this:

$ tar zxf latest.tar.gz
$ cd latest

In the directory you'll find a file called tables_init.mysql that could be used to create the tables used by the parser. Create a new database and make sure that the user you will use later do have the right permissions. The parser only make use of INSERT and SELECT queries.

$ mysql -u root -p your_database < tables_init.mysql
$ # Create a user with the proper permissions for the database

You also have to prepare the parser with information about your MySQL server, that is hostname, database, username and password. Open the file Parse.java in an editor. On the rows 40 through 43 you find the rows to edit. Here is an example:

	private static String serverName = "localhost";
	private static String mydatabase = "your_database";
	private static String username = "myuser";
	private static String password = "secret";

Save the file and exit the editor. Now we can build the parser using the script that comes with the package. When we have built the parser we make a test run on one of the smaller directories of the FreeDB.

$ ./build
$ java Parse /path/to/freedb/reggae

To fetch data from the database based on the DISCID you can run a query like this:

mysql> SELECT CONV(di.discid,10,16), d.length, d.title, d.year, g.name AS genre, t.track, t.title
    -> FROM discid_disc dd, discids di, discs d, genres g, tracks t
    -> WHERE di.pri=dd.pri AND dd.disc=d.disc AND g.id=d.genre AND t.disc=d.disc AND di.discid=CONV('010eb412',16,10);

Performance

The parser was developed on a AMD 64 3800+ with 3GB DDR RAM. During the tests the MySQL server was running on the same machine. To run the parser on the misc directory takes just over an hour, that's about 150 files every second. Considering that there are, in my version of the FreeDB, 1,927,902 files to parse you probably want to run the parser over the night since it will take about 3,5 hours to make a complete import.

License

The Viba IT FreeDB to MySQL parser.
Copyright (C) 2006 Viba IT Handelsbolag

This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.