Archive

Archive for September, 2009

Transactions & MySQL

September 29, 2009 Leave a comment

 

 

MySQL supports small, embedded kiosk-style applications as well as the occasional five billion-record data warehouse. This versatility is possible in part because of the MySQL engine, which has been designed for maximum scalability, maximum resource efficiency, and easy portability to various platforms and architectures. This section will discuss the important characteristics of this engine in detail.

 

Connectivity:

MySQL is designed on the assumption that the vast majority of its applications will be running on a TCP/IP (Transmission Control Protocol/Internet Protocol) network. This is a fairly good assumption, given that TCP/IP is not only highly robust and secure, but is also common to UNIX, Windows, OS/2, and almost any other serious operating system you’ll likely encounter.

MySQL does allow named-pipe connections, which were designed mainly to support network connections in earlier non-TCP/IP networks, such as LAN Manager and Windows NETBEUI. (NETBEUI uses an addressing scheme based on the NETBIOS machine name rather than a routable IP address.)

 

SQL:

The Structured Query Language (SQL) is an open standard that has been maintained by the American National Standards Institute (ANSI) since 1986. Although it’s true that the implementation of this standard does differ in varying degrees from vendor to vendor, it’s fair to say that SQL is today one of the most widely used cross-vendor languages. As with other implementations, such as SQL Server’s T-SQL (Transact-SQL) and Oracle’s SQL, MySQL has its own variations of the SQL standard that add power beyond what is available within the standard.

 

Data Integrity:

MySQL supports engine-level data integrity through the use of primary key and foreign key constraints. Columns can be defined so that explicit NULL values cannot be entered into them. To prevent empty columns, MySQL supports the use of default values, which, when combined with NOT NULL properties, ensure that valid data is entered into a column that would otherwise be left blank.

 

Transactions:

Until recently, MySQL was not known for its transaction-handling capabilities; however, since version 3.23, MySQL has been providing table handlers, such as InnoDB and BDB, that manage transactions in much the same manner as other commercial RDBMS products. A transaction-safe database system must pass what is known as the ACID test to qualify for compliance. An ACID-compliant database must support the following characteristics:

  • Atomicity
  • Consistency
  • Isolation
  • Durability

 

Atomicity:

A transaction is defined as an action, or a series of actions, that can access or change the contents of a database. In SQL terminology, a transaction occurs when one or more SQL statements operate as one unit. Each SQL statement in such a unit is dependent on the others; in other words, if one statement does not complete, the entire unit will be rolled back, and all the affected data will be returned to the state it was in before the transaction was started. Grouping the SQL statements as part of a single unit (or transaction) tells MySQL that the entire unit should be executed atomically.

Atomic execution is an all-or-nothing proposition. All of the SQL statements must be completed for the database to maintain a state of data integrity; otherwise, none of the statements will be finalized and committed to disk. In MySQL, the beginning of a transaction is marked with a BEGIN statement. The transaction (or unit of work) will not be considered complete until a COMMIT command is issued to tell MySQL to complete the action. When necessary, the ROLLBACK command will initiate a rolling back of all changes to the state before the BEGIN statement.

An everyday real-world example of this can be found in the banking business. By debiting and crediting your bank account, your bank adds and subtracts money from the account within one transaction. These updates usually involve multiple tables. The bank would not be able to maintain data integrity without guaranteeing that the entire transaction will take place, not just part of it.

Transaction management is particularly important to client-server systems that perform data entry or to any application that must be able to count on a high degree of safety from undetected data loss, such as the banking example described here.

 

Consistency:

Consistency exists when every transaction leaves the system in a consistent state, regardless of whether the transaction completes successfully or fails midway.

For example, imagine that your bank uses a transaction that is supposed to transfer money from one bank account to another. If the transaction debits one bank account for the requisite amount but fails to credit the other account with a corresponding amount, the system would no longer be in a consistent state. In this case, the transaction would violate the consistency constraint, and the system would no longer be ACID-compliant.

In MySQL, consistency is primarily handled by MySQL’s logging mechanisms, which record all changes to the database and provide an audit trail for transaction recovery. If the system goes down in the middle of a transaction, the MySQL recovery process will use these logs to discover whether or not the transaction was successfully completed and roll it back if required.

In addition to the logging process, MySQL also provides locking mechanisms that ensure that all of the tables, rows, and indexes that make up the transaction are locked by the initiating process long enough to either commit the transaction or roll it back.

 

Isolation:

Isolation implies that every transaction occurs in its own space, isolated from other transactions that may be occurring in the system, and that the results of a transaction are visible only once the entire sequence of events making up the transaction has been fully executed. Even though multiple transactions may be occurring simultaneously in such a system, the isolation principle ensures that the effects of a particular transaction are not visible until the transaction is fully complete.

This is particularly important when the system supports multiple simultaneous users and connections (as MySQL does); systems that do not conform to this fundamental principle can cause massive data corruption, as the integrity of each transaction’s individual space will be quickly violated by other competing, often conflicting, transactions.

Interestingly, MySQL offers the use of server-side semaphore variables that act as traffic managers to help programs manage their own isolation mechanisms. These variables are useful in cases for which you prefer not to incur the overhead of the Transaction Managers, or when a recovery plan is possible outside of the confines of log recovery. MySQL InnoDB tables offer isolation in transactions involving multiple queries, while MyISAM tables allow you to simulate isolation via the LOCK TABLES command.

 

Durability:

Durability, which means that changes from a committed transaction persist even if the system crashes, comes into play when a transaction has completed and the logs have been updated in the database. Most RDBMS products ensure data consistency by keeping a log of all activity that alters data in the database in any way. This database log keeps track of any and all updates made to tables, queries, reports, and so on. If you have turned on the database log , you already know that using it will slow down the performance of your database when it comes to writing data. (It will not, however, affect the speed of your queries.)

In MySQL, you can specify whether or not you wish to use transactions by choosing the appropriate table handlers, depending on your application. The InnoDB table handler performs logging a bit differently than BDB does, while MyISAM does not support the type of logs that would permit you to be assured of a durable database. By default, InnoDB tables are 100% durable to the last second prior to a crash. MyISAM tables offer partial durability—all changes committed to the system prior to the last FLUSH TABLES command are guaranteed to be saved to disk.

SQL Server and Oracle, for instance, are able to restore a database to a previous state by restoring a previously backed-up database and, in essence, “replaying” all subsequent transactions up until the point of failure. These database products do not encourage the direct use of—nor do they expose the inner data structures of—the log files, because those files form part of the database engine’s recovery mechanism.

MySQL also keeps a binary log of all data manipulation activity in sequential order. However, unlike the logs used in other databases, this log is easy to read, which means that it’s a relatively straightforward task to recover lost data by using the last backup in combination with the log.

 

Links:

http://www.mysql.com/

 

Note: Refer to other MySQL related posts for further knowledge.

 

Advanced Concepts:

Introduction:

Videos / Technology Talks:

What & Why?:

Installations & Configurations:

 

Advertisements

Tomcat Primer

September 18, 2009 1 comment

 

  

Tomcat Logo

 

 

 

Apache Origin:

Apache Tomcat (or Jakarta Tomcat or just Tomcat) is an open source servlet container started out as a reference implementation by Sun Microsystems and donated to the Apache Software Foundation (ASF). Tomcat implements the Java Servlet and the JavaServer Pages (JSP) specifications from Sun Microsystems, and provides a “pure Java” HTTP web server environment for Java code to run. Tomcat should not be confused with the Apache web server, which is a C implementation of an HTTP web server; these two web servers are not bundled together. Apache Tomcat includes tools for configuration and management, but can also be configured by editing XML configuration files.

Members of the ASF and independent volunteers develop and maintain Tomcat. Users have free access to the source code and to the binary form of Tomcat under the Apache License and hence is freely downloadable. The initial Tomcat release appeared with versions 3.0.x. Tomcat 6.0.20 is the latest production quality release of the 6.0.x trunk (based on 2.5 servlet specification).

Tomcat started off as a servlet reference implementation by James Duncan Davidson, a software architect at Sun Microsystems. He later helped make the project open source and played a key role in its donation by Sun to the Apache Software Foundation.

 

Tomcat Introduction:

Tomcat is a Java Servlet/JSP container and web server from the Jakarta project of the Apache software foundation. A web server dishes out web pages in response to requests from a user which typically is a web browser. But web servers are not limited to serving up static HTML pages; they can also run programs in response to user requests and return the dynamic results to the user’s browser. Tomcat is very good at this because it provides both Java servlet and JavaServerPages (JSP) technologies (in addition to traditional static pages and external CGI programming). Tomcat can be used as either a standalone product with its own internal Web server or together with other Web servers, including Apache, Netscape Enterprise Server, Microsoft Internet Information Server (IIS), and Microsoft Personal Web Server where the traditional server can be used to serve static pages and Tomcat can be used to serve dynamic servlet and JSP requests.

Tomcat is one of several open source collaborations called Jakarta. Tomcat requires a Java Runtime Enterprise Environment that conforms to JRE 1.1 or later. Developed under the Apache license in an open and participatory environment, Tomcat is intended to be a collaboration of the best-of-breed developers from around the world.

 

Some of the Tomcat and related definitions:

Apache:
“The Apache software foundation provides support for the Apache community of open-source software projects. The Apache projects are characterized by a collaborative, consensus based development process, an open and pragmatic software license, and a desire to create high quality software that leads the way in its field.” Amongst the projects that come under the “Apache” banner are the HTTP Web Server from which the whole Apache project has grown, and which is the container used for the majority of web sites worldwide, Ant (a build tool which allows the developer excellent control of the compiling and bundling processes), and Jakarta.

Jakarta:
“The Jakarta Project creates and maintains open source solutions on the Java platform for distribution to the public at no charge. Jakarta products are developed by and distributed through various sub-projects.” Jakarta is the name for the Apache project which deals with the provision of open source additions in Java. More than 20 such additions (known as sub-projects) are listed on their web site, including Struts and Tomcat.

Tomcat:
Tomcat is a servlet container for the Java Servlets and JavaServer Pages. It provides a Java Virtual Machine and associated elements to give a complete Java Runtime Environment, and it also provides web server software to make that environment accessible on the Web. Configuration and management tools are also provided, with configuration data largely held in XML. It’s worth noting that Tomcat is much more than just an implementation of Servlets and JSPs, it’s the official reference implementation and the standard against which all other suppliers of containers for Servlets and JSPs must measure their products. It means that developers know that if they develop code that works under Tomcat, that code should work under other containers that conform to the standards set.

 

 

Tomcat Architecture: Read more…

Why Tomcat?

September 16, 2009 Leave a comment

 

 

Why Webserver?

Have you ever wondered about the mechanisms that delivered this page to you? So, when you clicked on the link for this page, or typed its URL (Uniform Resource Locator), what happened behind the scenes to bring this page onto your screen? If you’ve ever been curious about the process, or have ever wanted to know some of the specific mechanisms that allow you to surf the Internet, then read on. At the most basic level possible, the following diagram shows the steps that brought that page to your screen:

 

 

 

webserver-basic

 

 

The browser breaks the URL into three parts:

  • The protocol (“http”)
  • The server name (“www.tecneosis.com“)
  • The file name (“why-tomcat.htm”)

The browser communicates with a name server to translate the server name “www.tecnoesis.com” into an IP Address, which then the browser uses to connect to the server machine. The browser then forms a connection to the server at that IP address on port 80. Following the HTTP protocol, the browser sends a GET request to the server, asking for the file “http://www.tecnoesis.com/why-tomcat.htm.” The server then sends the HTML text for the Web page to the browser. The browser reads the HTML tags and formats the page onto your screen. Your browser forms a connection to a Web server, requests a page and receives it.

 

Clients and Servers:

In general, all of the machines on the Internet can be categorized as two types: servers and clients. Those machines that provide services (like Web servers or FTP servers) to other machines are servers. And the machines that are used to connect to those services are clients. When you connect to Yahoo! at www.yahoo.com to read a page, Yahoo! is providing a machine (probably a cluster of very large machines),to service your request. Yahoo! then is a server. Your machine, on the other hand, is a user machine also known as a client.

The server usually serves either static or dynamic pages to the client. Static pages are those that do not change unless the creator of the page edits it. Dynamic pages are those which the server has to perform certain processes (invoke the component that executes perticular logic) to dynamically display the content to the client.

 

Tomcat as a Webserver: Read more…

Why MySQL?

September 15, 2009 Leave a comment

 

 

Graveled at commercial software licensing? Are you looking for a stable database platform but sweating at the hefty price tags attached to mainstream products from Microsoft, Oracle and other vendors? MySQL may be for you!  MySQL is a well-respected product that is more than capable of commercial operation. As of April 2009, MySQL offers MySQL 5.1 in two different variants: the MySQL Community Server and Enterprise Server, both having common code base and include the below features.

 

MySQL Features:

  • A broad subset of ANSI SQL 99, as well as extensions.
  • Cross-platform support.
  • Stored procedures.
  • Triggers.
  • Cursors.
  • Updatable Views.
  • True Varchar support.
  • INFORMATION_SCHEMA.
  • Strict mode.
  • X/Open XA distributed transaction processing (DTP) support; two phase commit as part of this, using Oracle’s InnoDB engine.
  • Independent storage engines (MyISAM for read speed, InnoDB for transactions and referential integrity, MySQL Archive for storing historical data in little space).
  • Transactions with the InnoDB, BDB and Cluster storage engines; savepoints with InnoDB.
  • SSL support.
  • Query caching.
  • Sub-SELECTs (i.e. nested SELECTs).
  • Replication with one master per slave, many slaves per master, no automatic support for multiple masters per slave.
  • Full-text indexing and searching using MyISAM engine.
  • Embedded database library.
  • Partial Unicode support (UTF-8 sequences longer than 3 bytes are not supported; UCS-2 encoded strings are also limited to the BMP).
  • Partial ACID compliance (full compliance only when using the non-default storage engines InnoDB, BDB and Cluster).
  • Shared-nothing clustering through MySQL Cluster

MySQL Enterprise Server is released once per month and the sources can be obtained either from MySQL’s customer-only Enterprise site or from MySQL’s Bazaar repository, both under the GPL license. The MySQL Community Server is published on an unspecified schedule under the GPL and contains all bug fixes that were shipped with the last MySQL Enterprise Server release. Binaries are no longer provided by MySQL for every release of the Community Server. Read more…

MySQL Primer

September 14, 2009 Leave a comment

logo-mysql

  

  

MySQL Origin:

MySQL was originally developed by Michael Widenius and David Axmark beginning in 1994, founders of a Swedish company MySQL AB. It was first released internally on 23 May 1995. Windows version was released on 8 January 1998 for Windows 95 and NT. Sun Microsystems acquired MySQL AB on 26 February 2008. MySQL AB yet holds the copyright to most of the codebase. In April 2009, Oracle Corporation entered into an agreement to purchase Sun Microsystems, current owners of the MySQL intellectual property. The deal was unanimously approved by Sun’s board of directors, and it is anticipated to close soon, subject to Sun stockholder approval, certain regulatory approvals and customary closing conditions.

MySQL is written in C and C++. MySQL runs on more than 20 different system platforms, including AIX, BSDi, FreeBSD, HP-UX, i5/OS, Linux, Mac OS X, NetBSD, Novell NetWare, OpenBSD, OpenSolaris, eComStation, OS/2 Warp, QNX, IRIX, Solaris, Symbian, SunOS, SCO OpenServer, SCO UnixWare, Sanos, Tru64 and Microsoft Windows; giving you the kind of flexibility that puts you in control. In addition, an ODBC interface called MyODBC allows additional programming languages that support the ODBC interface to communicate with a MySQL database, such as ASP or ColdFusion. The MySQL server and official libraries are mostly implemented in ANSI C/ANSI C++.

To administer MySQL databases one can use the included command-line tool (commands: mysql and mysqladmin). Also downloadable from the MySQL site are GUI administration tools: MySQL Administrator, MySQL Migration Toolkit and MySQL Query Browser. The GUI tools are now included in one package called MySQL GUI Tools. In addition to the these tools developed by MySQL AB, there are several other commercial and non-commercial tools available. Examples include Navicat Free Lite Edition, AnySQL Maestro Freeware Edition or SQLyog Community Edition, they are free desktop based GUI tools, and phpMyAdmin, a free Web-based administration interface implemented in PHP. 

  

MySQL Introduction:

MySQL is a relational database management system (RDBMS) which can be used to store, sort, arrange, and display information. MySQL stands for “My Structured Query Language”. The program runs as a server providing multi-user access to a number of database instances. The project’s source code is available under terms of the GNU General Public License, as well as under a variety of proprietary agreements. MySQL is commonly used by free software projects which require a full-featured database management system, such as WordPress, phpBB etc. It is also used in very high-scale World Wide Web products including Google and Facebook. Several high-traffic web sites (Flickr,Facebook,Wikipedia, Nokia, Auctionmarts and YouTube) use MySQL for data storage and logging of user data.

MySQL is an open-source database system with which we can do the following things:

  • Design the structure of the tables (called schema) and how they relate to one another
  • Add, edit and delete data
  • Sort and manipulate data
  • Query the database (that is, ask questions about the data)
  • Produce listings based on queries

To interact with MySQL, we enter commands on a command line. These commands, such as CREATE, INSERT, UPDATE, etc. are based on a more general language called SQL (Structured Query Language). SQL has a long and colorful history, starting at IBM in the 70’s (based on the research of E.F. Codd who developed the relational data model in 1970), and later through the work of a small company called Relational Software, Inc. In 1979, this group produced the first commercially available relational database system and implemented SQL as its query language. They called the product Oracle.

The MySQL database has become the world’s most popular open source database because of its consistent fast performance, high reliability and ease of use. Not only is MySQL the world’s most popular open source database, it’s also become the database of choice for a new generation of applications built on the LAMP stack (Linux, Apache, MySQL, PHP / Perl / Python.) MySQL represents the most impressive market success, exceeded only perhaps by Apache, in free and open source software. In terms of installed base, MySQL has left the technically impressive rival PostgreSQL behind. It has marginalized mSQL, SQLite, and SAP DB. It has started to challenge the proprietary database companies on their own turf. Nobody can say why licensing costs for proprietary databases have dropped sharply in recent years, but one suspects that it’s due to MySQL’s competition.

  • MySQL AB claims an installed base of five million systems, the largest of any database engine.
  • The mysql.com domain sees almost as much traffic as ibm.com.
  • Six hundred attendees flocked to the recent MySQl conference.
  • MySQL AB has recently been heavily marketing, its own publishing outlet,MySQL Press.

Many of the applications that a Web developer wants to use can be made easier by the use of a standardized database to store, organize, and access information. MySQL can easily be integrated into Perl programs by using the Perl DBI (DataBase Independent interface) module. DBI is an Application Program Interface (API) that allows Perl to connect to and query a number of SQL databases (among them MySQL, mSQL, PostgreSQL, Oracle, Sybase, and Informix).

  

  

Architecture: Read more…

Eclipse IDE Primer

September 13, 2009 Leave a comment

 

 

eclipse

 

 

Eclipse Origin:

In November 2001, IBM released $40 million worth of software tools into the public domain. Starting with this collection of tools, several organizations created a consortium of IDE providers. They called this consortium the Eclipse Foundation, Inc. Eclipse was to be “a universal tool platform — an open extensible IDE for anything and nothing in particular.” This talk about “anything and nothing in particular” reflects Eclipse’s ingenious plug-in architecture.

The initial codebase originated from VisualAge. In its default form it is meant for Java developers, consisting of the Java Development Tools (JDT). Users can extend its capabilities by installing plug-ins written for the Eclipse software framework, such as development toolkits for other programming languages, and can write and contribute their own plug-in modules. Language packs provide translations into over a dozen natural languages.

Released under the terms of the Eclipse Public License, Eclipse is free and open source software. Eclipse began as an IBM Canada project. It was developed by OTI (Object Technology International) as a Java based replacement for the Smalltalk based  VisualAge family of IDE products, which itself had been developed by OTI. In January 2004, the Eclipse Foundation was created. The Eclipse Foundation turned itself from an industry consortium to an independent not-for-profit organization. Among other things, this meant having an Executive Director — Mike Milinkovich, formerly of Oracle Corporation. Apparently, Milinkovich is the Eclipse Foundation’s only paid employee. Everybody else donates his or her time to create Eclipse — the world’s most popular Java development environment. According to IBM Chief Technology Officer Lee Nackman, the name “Eclipse” was chosen to target Microsoft’s Visual Studio product.

 

Eclipse Introduction:

 Eclipse is an open source, extensible, multi-language software development environment, comprising an IDE (Integrated development environment), and a plug-in system to extend it. At its heart, Eclipse isn’t only a Java development environment. Eclipse is a vessel — a holder for a bunch of add-ons that form, a Java, C++, or even a COBOL development environment. Each add-on is called a plug-in, and the Eclipse that you normally use is composed of more than 80 useful plug-ins. While the Eclipse Foundation was shifting into high gear, several other things were happening in the world of integrated development environments. IBM was building WebSphere Studio Application Developer (WSAD) — a big Java development environment based on Eclipse. And Sun Microsystems was promoting NetBeans. Like Eclipse, NetBeans is a set of building blocks for creating Java development environments. But unlike Eclipse, NetBeans is pure Java. So a few years ago, war broke out between Eclipse people and NetBeans people. And the war continues to this day.

 

 

Eclipse Architecture: Read more…

Why Eclipse IDE?

September 13, 2009 Leave a comment

 

 

If you’re reading this, you’re most likely a Java programmer, and you know how finicky Java can be at times. Missed import statements, forgotten variable declarations, omitted semicolons, garbled syntax, typos—all these problems will cause the Java command-line compiler, javac, to cough and display pages of annoying error messages. The error messages tell you that javac knows what the error is but doesn’t fix the problem. javac can’t fix the problem as it isn’t an editor. That makes long streams of errors scrolling off the page an all-too-common experience for Java developers, and leaves them with the feeling that Java is too prickly about what can go wrong. To change all that, you can use an integrated development environment (IDE), which will not only catch errors before you try to compile, but also suggest solutions. Java is badly in need of a good IDE, and a number of candidates are available, but the premiere Java IDE these days is Eclipse.

 

Open Source IDE:

If you closely follow open source or Java programming, you should have heard the buzz about Eclipse. Eclipse is an extensible software development environment comprising an IDE (Integrated development environment – It is an all-in-one tool for writing, editing, compiling, and running programs) and a plug-in system to extend it. 
 
It is written primarily in Java and can be used to develop applications in Java. And by means of various plug-ins, it can be used to develop applications in other languages as well, including C, C++, COBOL, Python, Perl, PHP, and others. Eclipse employs plug-ins in order to provide all of its functionality on top of (and including) the runtime system. The runtime system of Eclipse is based on Equinox – an OSGi standard compliant implementation.

 

Plugin Extension in Eclipse IDE: Read more…