Empyrean Age Data Export | EVE Online

Empyrean Age Data Export

2008-07-30 - By CCP Prism X

Until now there hasn't been a clearly defined process for releasing the data export after each expansion, and it has usually fallen on the shoulders of individual developers to sort it out and release it. The last data export was released by CCP Garthagk after Revelations 2 and, prior to his involvement, CCP t20 was responsible for it here and here. After a handful of releases, we concluded that better tools and processes were needed to allow our Quality Assurance and Deployment groups to easily create and release these data exports. This makes them more easily available for other EVE projects, such as EVElopedia that will, among other things, include a new and improved Item Database. The task of creating those tools wound up on my desk.

With a new developer comes a new approach. The previous data exports, both by CCP Garthagk and CCP t20, were in the form of text files with table definitions and insert statements for the data. That method isn’t as maintainable as we would desire and instead our future exports will be in the form of MSSQL database backups (.BAK), which third party developers can restore on their machines to extract the data into whatever form their application uses. If you do not have any available methods of working on MSSQL databases then try SQL Server 2005 Express Edition, as it has the nifty pro of being free.

The new data export is downloadable from the EVE Online website, accessible from the Community part of the website, or from here.

Along with the backup itself you will get a .SQL file called ebs_DATADUMP.sql which will detail the table definition along with existing indexes and cluster definitions. We have tried to keep it in line with the older dumps to avoid excessive refactoring for third party applications. The most notable exceptions are:

  • Addition of table agtResearchAgents which will detail what agents have what research skills.
  • Renaming of table TL2MaterialsForTypeWithActivity to typeActivityMaterials.
  • Inclusion of primary keys and indexes.
  • Removal of table crpNPCCorporationTrades.

It is not our intention to publish all future data exports with Dev Blogs so any changes to ebs_DATADUMP.sql, be they additions, removals, renaming, or what have you, will be detailed at the top of ebs_DATADUMP.sql, preceded by a date of change and/or version.

As a final note, in the past we've had requests for inclusion of foreign keys. Fact is that we have no proper foreign key definitions for technical reasons so there are no foreign keys to include. However, any column referenced by other columns usually either has the same name as their referencing column or contain the name of their referencing column as a suffix preceded by a further explanation of the column.

For example, agtResearchAgents.agentID and agtConfig.agentID reference agtAgents.agentID, also typeActivityMaterials.typeID and typeActivityMaterials.requiredTypeID reference invTypes.typeID. Of course nothing is truly that simple as there are some more obscure examples of references like mapDenormalize.orbitID, mapJumps.stargateID and mapJumps.celestialID all reference mapDenormalize.itemID and mapJumps.celestialID actually references a stargate (mapDenormalize.itemID) as we consider them celestials.

In addition to a robust way to create the data export, we needed an automated way to render all the module and alliance icons, 3D models, and other miscellaneous imagery. The system behind the scenes has been rewritten quite a few times to enable a fairly solid degree of sturdiness.

What we ended up with was an automated way to render out any level and grain of detail for every single icon you can see in-game – with certain size restrictions of course – as not every icon is "rendered" but merely exported from an already existing texture.

The icons will be available for download on the same web page as the data export.

CCP Prism X

Update: You can download a more recent version (2008) by following this link.