|
Intelligent Convertors
Searching the internet for database converters and their functionalities, some of
database converter software or tools do not distinguish between:
• Data Migration
• Data Conversion
Data Migration:
Data migration or Data Center Migration is the process of deploying and transferring an existing
data center environment to another data center operating environment. Migrations include
hardware, software, storage, network equipment, cooling systems, power supplies and data.
See our Post Office Project Migrating Data Center Using Object Oriented (OO).
Data Conversion:
Data conversion on the other hand is the conversion of computer data from one format to another.
Our Database Convertors:
Our database convertors are software which convert any existing database (tables, fields and files) and
any existing structured or unstructured (XML, PDF, spreadsheets, text, forms, ... etc) data into our new Intelligent Data Access Objects (IDAO).
Our Converters also create Bit, Indexes, hashing matrixes and a limited number of database tables. This new
format with its structure and objects are faster-intelligent-economical-reusable ways for processing
Big Data. The new structure and objects can be added to any database. In case of Data Center
Migration or Building, they are ideal for the creation of the new format into Data Centers.
The goal is to eliminate-minimize the use of databases tables and create the following:
•
|
Retrieve every field in any database and parse and converted into our IDAOs
|
•
|
Parse any existing structured or unstructured (XML, PDF, spreadsheets, text, forms, ... etc) data into our new IDAOs
|
•
|
Eliminate the use of schemas for parsing XML tags-files and use instead XML primitive
data types which are compatible to Java primitive data types plus the use of tags to
represent Java objects
|
•
|
Matrixes are created and used for faster parsing, conversions, search and analysis
|
•
|
IDAO (Java objects) which would be stored in the database as XML files
|
•
|
In the case where the databases do not support XML fields, they can be stored as CLOB, BLOB fields
|
•
|
Limited number of Lookup Tables and audit trials
|
An Example:
Retrieving every field in one of the airlines' databases and parsing them is a monumental task, therefore
we need to develop a scheme of retrieving and categorize each field.
Our approach is to create one or more index for each possible field and its value with an ID.
IDAO Array - Figure A
* This index (secondary key) is the field address of the parsed field.
The best way of illustrating our approach is with an example. For simplicity, let assume
that we have:
Name:
Mr. "X"
ID no.:
"001"
Using an airline database fields and based
on the database field type and dynamic business rules, we are able to create the array
in figure A for Mr. X. This array can be used to create our IDAOs:
•
|
The array has Personal, Business, Transaction, Security and Misc columns.
|
•
|
Each column would be corresponding to an IDAO type which our converter would
create a corresponding Java IDAO for Mr. X
|
•
|
We used "N.A." not applicable if such field does not apply to the IDAO type (Business Rules)
|
•
|
The "Index Number" column is the secondary key of the mapped addresses of the parsed database
|
•
|
IDAO Java objects can retrieve the field values using the Index ID
|
•
|
All of Mr. X Java IDAOs would be converted to XML format
|
•
|
No "Schema" is needed in this conversion since both Java and XML have the same primitive
data types and XML tags can correspond to Java IDAOs properties-variables
|
•
|
All of Mr. X Java IDAOs can be stored in one XML file which would be stored in
one XML database field or as CLOB or BLOB field in the database
|
•
|
Using our IDAO approach simplifies Extract, Transform, Load (ETL) to one
read (XML file) into memory for use (cashing- memory resident) and one write (XML file)
to the database to store and update and no more tables and fields
|
The following is what Personal IDAO would stored for the "Name Field" for Mr. X:
1PS001
1 = Index ID "1" for retrieving the actual field value
P = Personal Data (this value can also be a number instead of a letter)
S = Java String type (this value can also be a number instead of a letter)
001 = Mr. X ID
This array bucket value can be used in building the java IDAOs class properties or variables.
From this array, our converter can create different IDAO (Java class) and their properties since
we have the name and data type and index to where to retrieve its value from the database using "Index Number".
This array is the map for our converter to start converting database fields into IDAO and groups
them into XML files to be stored in the database.
What we just presented is the seed for building an automated Intelligent Parser-Converter which
would create a new format for every database field.
|
|