|
Our Intelligent DevOps Editor - "Turn Key" Software
The DevOps goal is to improve the communication and cooperation between the development and the
infrastructure teams. Sadly, nothing is mention about the infrastructure needs for other departments'
teams such as Sales, Marketing, Credit, Treasury, etc. These teams have web presence and run third
party and customized software tools. Building Intelligent DevOps Editor with "Turn Key" features
requires that we must have a good handle on all the details of running infrastructure team (provider) and
their clients. To have "Turn Key" features, we must address the needs of our editor end-users.
"Turn Key" features would reduce if not eliminate any or all the interfaces
between the infrastructure team and their clients. We also need to leverage Virtualization in
building virtual networks. The following is our attempt to give the reader a view of how we would
build our Intelligent DevOps Editor:
Existing Processes, Structures and Issues:
Existing Infrastructure Processes, Support and Tools
Infrastructure Building-Migration's Time, Effort and Constrains
Scrum Development Processes and The Goals of DevOps Approach
Other Departments' Infrastructure' Needs
Using Virtualization:
Virtualization is The Key
Virtual Servers, Virtual Server Containers and Virtual Networks
Our Approach:
Object Oriented Design (OOD)
The Concept of "Turn Key"
Our Intelligent DevOps Editor
DevOps Editor GUI Interface
Our Intelligent Infrastructure Editors:
What is already done - Our Intelligent Infrastructure Editors
Pros and Cons
Existing Processes, Structures and Issues
Existing Infrastructure Processes, Support and Tools
The following is a quick summary of the infrastructure architect's processes, support and tools:
|
Infrastructure Architect Engineer (IAE) is responsible for architecting, building, managing and maintaining the entire
software (clusters, virtual and bare metal servers) infrastructure and data centers.
|
|
IAE works with Development, Testing, System Infrastructure, Sales Infrastructure
and the rest of institution's departments.
|
|
IAE works with other departments teams to create projects' Technical Requirement Document (TRD)
and Non-Technical Requirement Document (NRD), Proof of Concept (POC) and other Infrastructure design docs.
|
|
IAE's tasks include analysis, architecting, documenting and tracking the building
and the execution of the requested Infrastructures.
|
|
IAE creates specs-docs using a number of editors and software tools.
|
|
Infrastructure docs-specs must be approved
|
|
These Docs-specs are given to offshore teams to build and test
|
|
IAE would trace, test and document the offshore work and approve them
|
|
The infrastructure design is prepared for delivery
|
|
IAE handles any change requests and may perform a number of the listed processes.
|
|
Infrastructure software tool used: IIS, Apache, Weblogic, JVM, Tomcat, JRE, JDBC,
Oracle, EXADATA, Netezza, NAS, IBM MQ, SFTP, NDM, SAN, ETL Informatica, Unix, Redhat Linux, Solaris, Windows, Golden Gate.
|
Infrastructure Building-Migration's Time, Effort and Constrains
The effort, the time and the documentation for delivering any infrastructure component take
months with the never-ending "change Requests". The Infrastructure teams are structured in layers which start with Infrastructure
architect Engineer (IAE) performs the architect-design, specs, and documentation and
end with offshore teams that performs the actual installation and testing. Micromanagement
is done to insure no issues or errors. The inter-departments politics also play an additional
burden that most institutions suffer from.
The cost factor here is also substantial and we would not cover it in this page.
Scrum Development Processes and The Goals of DevOps Approach
The Infrastructure team can provide the following:
Browser-Mobile software Support , External Interfaces, Load balancers, Firewall, File
Server, File exchange, NAS, SAN, Batch, Documentation, Internal Interface, Third-Party
software, Products, Database Internal Interface, Web servers, Application servers,
Database servers and Bridge.
These features or support go through analysis, design, approval, building, testing and delivering.
Scrum methodology:
The Scrum idea is to make the development life-cycle incremental and iterative. Incremental
means working in small chunks at a time. The team works on few requirements at a time, turns it into
working software and only then starts working on the next set of requirements. Iterative
means taking feedback periodically and incorporating the lessons from previous iterations while planning future work.
Testing and Production Environments:
Development teams can use their desktop, laptops, source safe tools and local servers to perform
the Scrum incremental approach. The development team must communicate their timeline requirement
to infrastructure team. Their requirement must include what and when testing and production servers
should be available.
Change Request:
Change Request is a separate issue which needs to be addressed early in the communication and cooperation.
The Goals of DevOps Approach:
What exactly is the concept of DevOps?
DevOps (development and operations) is an enterprise software development phrase used to
mean a type of agile relationship between development and IT operations. The goal of DevOps
is to change and improve the relationship by advocating better communication and collaboration
between these two business units.
The number of the DevOps tools in the market are quite a few and they promise the world. We need to research
these tools and at this point in time we cannot make any judgment.
Other Departments' Infrastructure' Needs
Any large corporation such as big banks would have several departments and each may have their
own software and tools which would require infrastructure support also. Regardless if the Agile-Scrum
approach is used or not, the same issues of communication and cooperation would still exist.
Using Virtualization
Virtualization is The Key
A Network is all the running hardware, software, interfaces, wiring, IP addresses, licenses
and anything any network requires.
Network virtualization (NV) supports the complex requirements in multi-tenancy environments. NV can
deliver a virtual network within a virtual environment with independent network
resources. These NV can disperse traffic into zones or containers to ensure traffic
balance and resources distribution. Customizing virtual servers and networks to meet the business
requirement is the way to go
Virtual Servers, Virtual Server Containers and Virtual Networks
What is a server?
Server software is a type of software that is designed to be used, operated and managed
on a computing server. It provides and facilitates the harnessing of underlying server
computing power for use with an array of high-end computing services and functions. In
short, a server is a functional-programmable software components.
What is virtual server?
A virtual server (VS) is a server that shares hardware and software resources with other
operating systems (OS). Because they are cost-effective and
provide faster resource control, VS(s) are popular in Web hosting environments.
Each VS can run its own operating system and applications. Therefore, virtualization reduces
the need for additional server infrastructure and provides a cost-effective way to deliver
different types of services
Ideally, a VS mimics dedicated server functionalities. Rather than implement
multiple dedicated servers, several VS may be implemented on one physical
or VS. Therefore, a VS may contain other VS.
Can building VS be automated with GUI tool?
A VS is software which can be installed and configured.
Scripts and Templates:
Templates are used as building blocks for creating frameworks or projects. On other
hands, scripts are used to start running software, tests and deployments. With
same principle, we can use template and scripts to build and run VS.
Tools for building VS:
Searching the internet looking for automated tools to build VS, sadly we
need to find out which tool is best for our GUI automated Virtual Builder Editor.
Can a GUI tool be created to build VS?
The answer is yes.
Our Approach
Software tools are bought "as is" from vendors, build-in-house, outsourced to consultants or
combination of buying, building and outsourcing. Most institutions such big banks or insurance companies
are not in the business of building software, but they still have several development and testing
departments. The same thing can be applied to infrastructure. Customization and support of these
software tools may run into several issues such as cost, time and changing technologies. These
institutions would be under the mercy of the needed support from their software vendors and consultants.
In-house tools have very high risks if they are not well designed, tested and
documented. Therefore, our approach is to build a well designed, well tested and
well documented software tools.
Our approach is to use Object Oriented Design with Virtualization to build our Intelligent DevOps Editor.
Object Oriented Design (OOD)
Object Oriented Design (OOD) can be used to build a virtual structured environment. It would be composed
of manageable objects. Our OOD Virtual Model (see the link of Building Using OOD web Page) would make it easy to:
Structure and size
Build
Copy
Clone
Modify
Test
Inherit
Move around
Also such virtualization can be created as a running prototype, or we can build software programs
as a model virtualizer similar to ones that are used in building airplanes, big buildings or
high-rises. Such a program(s) can be created as a way to give estimates, be part of the planning as
well as test other criteria without building anything.
For more information on our virtual OOD approach see the following link:
"http://crmdatafarm.com/BuildingUsing_OOPage.html"
Image #1 is a simple representation of how Virtual Servers
and Virtual Servers Container can be implemented. Infrastructure teams can work with a software
vendor to develop their own unique and customized software tool with GUI interfaces. This would be used to
automate building Virtual Servers and Virtual Servers Containers.
Image # 1
Figure #1 gives a graphic representation of building Object Oriented Servers and Server
Containers. The Application Server Container is a virtual object which has other virtual
servers as virtual objects. An Application Server Container may have a virtual web server
such as Tomcat, a firewall software as virtual server and virtual File Transfer Server such
as FTP. The same thing can be applied to the Database Server Container with virtual FTP
server, Oracle virtual server or any third party software running as a virtual server within
the Database Server Container.
With such server virtualization, then automation and customization can be implemented. Such
automation can produce an infinite number of virtual servers and containers. These virtual
servers can be created, customized, installed, duplicated, cloned, deleted or cut/paste into
and out of any network or virtual network.
The Concept of "Turn Key"
Turn Key of or involving the provision of a complete product or service that is ready for immediate use.
The goal of our Intelligent DevOps Editor is eliminate the communication and the cooperation issues
between the Infrastructure team and their clients such as the development team. Our Editor provides
a GUI interface for any team to create any server type and software installation for their infrastructure
needs. They simply use an interactive-user-friendly GUI interface (see Image #2) to fill out all specification
parameters. Our Editor parses their requests and runs a number of checks which include resource, security
requirements, time of delivery and the team access and resource privileges. It may requires further
questions, ID and permission privileges. It would grant the team's request with a un-shaded GUI "Build"
button which would be clicked to start in real-time building of all their requested infrastructure servers
and software. They would also be able to run a number build-in tests to insure their approval.
With the same GUI interface, teams can save, save as, delete, clone, inherit previous builds, and other options.
Roll Back can be easy as build one of the previous saved version or setting.
The backend of our editor has a number of software components which would build in real-time all the
infrastructure components and their tests.
A number of statistical software components run to help with updating and evaluating the editor performances.
Our Intelligent DevOps Editor
What is a software editor?
In general, an editor refers to any program capable of editing files. Good examples
are image editors, such as Adobe Photoshop, and sound editors, such as Audacity or
for text editor such as Microsoft Word or WordPerfect.
What is a DevOps Editor?
The best answer is to present a picture or image and Image #2 is our Editor's markup or prototype for
audience to examine. The editor features are covered in the DevOps Editor GUI Interface section.
How can we build such an Intelligent DevOps Editor?
The key feature of our editor is Intelligence. Intelligence here is not Artificial
Intelligence, but developing intelligence software. We communicate with gurus of
development and infrastructure, and try to pick their brains. We build a number of
processes and tasks mimicking these gurus handling and approaches. We rearrange
these processes and tasks in order to be able to translated them to code and a running
software. With the computer processing speed, thousands if not hundreds of thousands of
processes and options can be performed on the input data in seconds. These processes
add to intelligence of our editor.
The following are the steps and processes which would include the analysis, architect-design
and development, testing and documenting of building our editor. It also includes running
statistics for improving our editor and its performance:
Interview all the gurus of every business unit and pick their and turn their expertise into processes
Research other companies and competitors for building networks
Figure what the outside world is doing
Create a set of questions and answers that would be asked the end-users
Interview the end-users and figure out what is nice to have and have to have
Document best practice for build a virtual server
Research best automated virtual server builder software-tools in the market
Run a number of statistics to find out frequent settings built by each department
Build a number of default setting and values
Create a number of automated tests for each virtual server setting
Run a number of statistics to find most errors and how they would be handled
List all possible misc handling processes
List all possible exceptions and errors handling processes
Build a number of script, tests and templates which can be used in our build
Estimate the cost of building the editor
Brainstorm the editor with the team and management
The above steps and their output are our editor architect-design-development-testing specs.
We would build the editor markup and prototype and test them with end-users feedback.
A "Beta" version should be developed and tested with end-users feedback.
DevOps Editor GUI Interface
To make our Intelligent DevOps Editor concept easier to understand, we may need to
present a picture. We are presenting a markup or a running prototype with a GUI
interface for our audience to examine. Image #2 is our editor prototype and
also a link to an HTLM page which would popup for our audience to play with and
see our editor's features. For further clarification, we are presenting a summary
of some of our editor' features or functionalities.
Image # 2
Intelligence:
Check previous section which covers Intelligence: "How can we build such an Intelligent DevOps Editor?"
Input Data:
Our Editor's users would enter all the required data which would be stored into Java arrays
and objects. Our Editor's backend processes would cross reference the data looking for
errors, duplicate values, out of range values or any red flags. It
would prompt the users for corrections and what is doable.
Check List:
Users can click the "Check List" button to make sure that all the required data are entered.
Test, Clone and Build:
Users have the option to test their input data and see if our editor can
create a running target environment. They can "Save" or "Save as" the entered data
and settings. They can import from other previously built environments. The difference
between "Clone" and "Save as" is "Save as" saves the input data, and "Clone" on the
other hand would save the running environment with all its updated settings and
scripts. As for Build option, our editor would build the target environment for
immediate use in real-time. Such running environment can be cloned as well a saved.
Default Settings:
Our Setting is not just a random settings, but we would be using the history of
each department (development, credit, Sales, etc) infrastructure features, settings
and issues. Project name and ID would identify which department is using our editor. Based
on which department is using our editor, the default setting would be automatically
loaded into the editors. Each department would have its unique default settings.
The default setting will change as technologies, users demands and Changes Requests.
Documentation:
To us documentation is very critical to our running environments. Documentation
can help in tracking and finding errors, issues and trends. It can also aid in
building statistic which would be used to make our Intelligent editor more intelligent
and dynamic. To automate documentation, we build two documentation sections:
Build-in Templates: which would be filled out by the users.
Free Hands: where users to give their own document and feedbacks.
Users' free hands would be periodically checked for how we can improve our editor.
Management:
Management is another critical part of running and managing these target
environments. Again, we have two management sections:
Management Processes: which would help managers perform their expected tasks.
Free Hands: where managers give their own document and feedbacks.
Managers' free hands would be periodically checked for how we can improve our editor.
Online Help Doc and Recommended Settings:
Our Editor users can be anyone from any department. Users may or may not be network
or web savvy, nor expected to experts on infrastructure requirement. Online Help
Doc and Recommended Settings are the documentation of how to build the required
environments with examples and the default settings. We would also provide tutorials
and training for departments team members.
Target Environment Software Components and Environment Settings:
Users may use the default setting or change them to their
needs. The documentation for the online help and setting are for users to utilize
in figuring out their environment needs.
Testing:
Testing is very critical in building error free environments. There is auto testing and manual testing.
Manual testing is an option for users to test their own unique scenarios.
Our Editor would not allow any Building unless auto testing or manual testing are completed.
Our Intelligent Infrastructure Editors
What is already done - Our Intelligent Infrastructure Editors
We recommend that our viewers visit our CRM Data Farm site and checkout our Post Office
Project's Intelligent Infrastructure Editors for Building, Migration and Testing Data Centers.
For Our Intelligent Infrastructure Editors see the following links on our CRM Data Farm site:
http://crmdatafarm.com/
http://crmdatafarm.com/OO_DataCenters.html
http://crmdatafarm.com/BuildingUsing_OOPage.html
http://crmdatafarm.com/MigrationUsing_OOPage.html
http://crmdatafarm.com/TestingUsing_OOPage.html
Pros and Cons
Our ICM architect is not small project nor easy to implement. Our Intelligent
DevOps Editor is a tough sell to any company.
Pros:
Makes ICM's goal of building intelligent data services a reality
Remedies the communication and cooperation
Can be used by other departments not just the development
Sets the stage for more intelligent and futuristic approaches
Helps in building data centers and reduces the cost of building and migration
Cons:
Needs a good grasps on subject matter
Needs dedicated teams of analysts, architects and developers
It will be a hard sell to management
|
|