Confessions of a Java Programmer

Advanced Web Application Test Development Using Selenium IDE

2010-07-12T00:01:00.000-07:00

In my last post I covered creating a basic Selenium test using the Selenium IDE and exporting tests to JUnit for execution through Selenium RC. Now, we're going to cover the more advanced aspects of using Selenium IDE to create advanced test cases for your we application.

The easiest way to generate tests, is too perform the actions in your browser and allow the Selenium IDE to populate automatically. In all honesty, this will handle 99% of the features you want to test. But there are times when the browser integrated features aren't enough to test everything and you have to get your hands a little dirty.

Assert the Title Attribute of an Image

For our first example, we're going to verify the title attribute of an image. One of my favorite sites is XKCD and the comics always have a funny image title that is part of the comic. The first thing we're going to do is browse over to http://www.xkcd.com/756/. If you hold the mouse over the image, you will see the title text appear, which says: "News networks giving a greater voice to viewers because the social web is so popular are like a chef on the Titanic who, seeing the looming iceberg and fleeing customers, figures ice is the future and starts making snow cones."

Let's fire up the Selenium IDE by opening Tools > Selenium IDE. Set the base URL to http://www.xkcd.com. Now if you'll right click on the comic, then go to Show all available commands > assertElementPresent //img[@alt='Public Opinion'].

The assertElementPresent command allows you to specify an element exists using locators. We're going to cover locators next. Switch over to the Selenium IDE and select the assertElementCommand we just created. In the target text box, change the @alt='Public Opinion' value to @title=News networks giving a greater voice to viewers because the social web is so popular are like a chef on the Titanic who, seeing the looming iceberg and fleeing customers, figures ice is the future and starts making snow cones.

This means that we are going to locate the image based on the title attribute rather than the alt attribute. Now your test should look like my test in the screenshot. Run the test now by clicking on the green triangle withe three bars and the test should pass. We now have a test that ensures the title text for this particular web comic is what we think it should be.

Selenium Element Locators

Element locators within Selenium allow you to specify which element on a page is the target of a given command. There are three default locators for Selenium that don't require you to specify what type of locator you are using:

DOM
Locators starting with "document" will use the DOM Locator.
XPATH
Locators that start with "//" will use the XPATH locator to find a given element.
Identifier
Locators that start with anything other than "document", "//", or a valid locator type, will default to using the identifier strategy. When using the identifier strategy, the first element with a matching id attribute will be selected. If no matching id is found, then Selenium will use the first element with a matching name attribute.

Web Application Testing with Selenium and JUnit

2010-07-03T13:01:00.000-07:00

Selenium is a suite of tools to automate web app testing. Selenium is very flexible and integrates with many testing frameworks and languages as well as running within many operating systems and browsers. The three major parts that make up Selenium are:

Selenium IDE
Integrates into Firefox as an add-on to allow you to generate tests that can then be exported out to other formats for automated testing setups. This post will cover using the Selenium IDE to generate tests to be used with JUnit.
Selenium Remote Control
Runs the test generated by the IDE on various platforms and browsers allowing for robust automated testing of your web application. Selenium RC will be lightly covered in this post, but will be covered in more depth in a later post.
Selenium Grid
Builds upon Selenium RC and allows you to distribute your tests across multiple servers for parallel execution. Selenium Grid will be covered in a later post.

Creating your First Test with Selenium IDE

The first thing you'll need to do is install the Selenium IDE add on into Firefox by going to http://seleniumhq.org/download/. Once the Selenium IDE has been installed, you'll have to restart Firefox and you'll be ready to begin creating tests.

Start Firefox and click on Tools > Selenium IDE to open the IDE so you can begin creating a new test. Click the record button inside of the Selenium IDE dialog, the red circle button, then you can begin performing the actions that will make up your test. This process is pretty much just browsing to the application that will be testing and being specifying what should be tested.

For our first example, we're going to use the Selenium IDE to build a basic test that asserts the title is correctly set for http://javaconfessions.com. We want to verify not only that the title is set correctly in the head of the document, but we want to make sure the title is showing properly at the top of the page.

Ensure that we have clicked the record button, the red circle button, then in the browser go to http://javaconfessions.com. Once the page loads, right click anywhere on the page and hover your mouse over the last item in the context menu which will be "Show all available commands."

Once the flyout menu appears, click the "assertTitle Confessions of a Java Programmer" menu item. You will see this command populate into the Selenium IDE. This command will assert that the title inside the head of the main HTML page reads "Confessions of a Java Programmer."

Now that we've tested to ensure the title in the head tag is correct, we're going assert the title also shows up on the top of the page. This time, select the header text at the top of the page that reads "Confessions of a Java Programmer." Right click and once again hover over the "Show all available commands" menu item, then click the "assertText //div[@id='header-inner']/div[1]/h1 Confessions of a Java Programmer" menu item. This command will assert that the top of the page reads "Confessions of a Java Programmer."

Running the Test in the Selenium IDE

Now that we've gone through the process to setup a basic test, we're going to run it in the Selenium IDE in order to show that it works the way we think it should. To execute the test in the Selenium IDE, click the button that has the green triangle with three bars to execute the entire test.

The browser will go through all of the actions defined in the test and it will pass. If there is a failure, you will see a red message in the output at the bottom and then you know you need to rework your test to ensure you are getting accurate results.

Exporting the Test for JUnit

Now that we have built the basic test in the Selenium IDE, we can export our test for use in a JUnit Test Case. In the Selenium IDE go to File > Export Test Case As > Java ( JUnit ) - Selenium RC. Name the file JavaConfessionsTestCase.java.

Open up the newly generated test file and change the package for the class if you desire. By default the test class will be created with a package of com.example.tests. After you are done editing the file close it and copy it into the appropriate package in your project's testing directory.

Running the Test

Now we need to download Selenium RC from http://seleniumhq.org/download/. Unzip the newly downloaded file and copy out the file named selenium-java-client-driver.jar out of the selenium-java-client-driver-1.0.1 directory to your testing environment's class path.

The next step is to copy out the selenium-server.jar out of the selenium-server-1.0.3 directory to a permanent spot on your testing environment server. This file can be put anywhere, as it will be ran as a service in order to allow Selenium testing from JUnit classes.

Before executing our JUnit test we need to start the Selenium RC server. Open a command prompt and change into the directory where you stored the selenium-server.jar file. Then start the Selenium RC server by typing java -jar selenium-server.jar.

With the Selenium RC server now running , you can execute your Selenium JUnit test just like you normally run JUnit tests. In the screenshot, you can see the results from executing the Selenium test in IntelliJ IDEA. In the next post, I will demonstrate how to create more complex Selenium tests.

Add Rows to Groovy's Swing Builder DefaultTableModel

2010-02-11T15:39:00.001-08:00

If you've had an opportunity to mess around with Groovy's Swing Builder, you may have noticed that the table model used for tables is not the standard javax.swing.table.DefaultTableModel, but is an instance of groovy.model.DefaultTableModel. Java's implementation of the DefaultTableModel has an addRow() method that allows the addition of new rows, however Groovy's implementation does not.

This left me, and many others, wondering how to go about adding a new row to Groovy's DefaultTableModel object. After looking over the Groovy API, I have come up with the solution below. Enjoy:

import groovy.swing.SwingBuilder

def swing = new SwingBuilder()
def tableData = [ [firstName: "Bob", lastName: "Stevens"], [firstName: "Lucy", lastName: "House"] ]
def myTable = swing.table(){
tableModel( list:tableData ){
propertyColumn( header:"First Name", propertyName:"firstName" )
propertyColumn( header:"Last Name", propertyName:"lastName" )
}
}

// Let's add a new row to the table
def rows = myTable.getModel().getRowsModel().getValue()
rows.add( [ firstName: "Reese", lastName: "Smith" ] )
myTable.getModel().getRowsModel().setValue( rows )
myTable.getModel().fireTableDataChanged()

This example will add a new row to the end of the table and then redraw the JTable so the new row will show up. The function getRowsModel() returns the ValueModel which allows you to manipulate the data within the table. The value set within the ValueModel will be a list of the rows in the table and you can add rows by calling add( rowData ) or specify an index with add( index, rowData ) to allow you to insert data at a specific spot in the list.

Fix Slow Charter DNS Lookups by Using OpenDNS

2010-02-02T04:52:00.001-08:00

I have been having issues with DNS resolution being extremely slow with Charter Communications Internet on my Macbook. I tried disabling IP v6 in Firefox and that seemed to work for a little while, but eventually my DNS lookups slowed down again. Frustrated, I began looking for ways to not use Charter's DNS servers.

I found out about OpenDNS after googling a little bit. I opened up a free account and switched my router over to their DNS servers. Low and behold, my Internet speed is blazing again on my Macbook. So, if you're having issues with slow DNS lookups on OSX and Charter is your ISP, give OpenDNS a try.

SmartSVN Blows the other Subversion Clients out of the Water

2010-01-21T07:25:00.001-08:00

I was a developer in a Window's environment for many years and have had a few years of experience with TortoiseSVN. Once I switched to Ubuntu a while back, I strugged to find a suitable replacement for TortoiseSVN in Linux. I tried a few different solutions including RapidSVN and KDESVN. Neither of those even came close to the ease of use and stability of TortoiseSVN.

A friend who develops on OS X had mentioned that he uses SmartSVN and that it is available for Linux also. As a matter of fact, you can run SmartSVN under Windows, OS X, Linux, and even OS/2 Warp.

So I downloaded it and ran through the 30 day professional version trial. At the end of the trial I immediately paid the fee for the professional version. I know what you're all thinking. This sucker paid $79.00 for a Subversion client when there are a billion free ones available. I gladly paid the fee and say with great confidence that there is nothing out there that even comes close to what SmartSVN can do for you.

Even if I was still developing in a Windows environment and could still use TortoiseSVN, I would have paid the fee to use SmartSVN. TortoiseSVN does not even come close to the level of productivity I can accomplish under SmartSVN.

Merge Single Revisions Across Branches

I work in an environment with many different branches. At any given time I have to work on the trunk and two or three branches. If there is a change that needs to go into the trunk and possibly one or two branches, it is a time consuming process to get the changes everywhere they need to be.

Assuming I am using TortoiseSVN, I'll have to commit the changes to the trunk. Then using a diff tool, merge the changes I made from the trunk to the branch. It is all very manual and error prone. TortoiseSVN does offer a merging tool, ~~but this is only to merge the entire branch, not single revisions or files~~ but it is uselessly complicated in my opinion. You can read more about TortoiseSVN's merging functionality here.
Using SmartSVN, this same task is so simple. After you have commited your trunk changes, switch over to the branch where you wish to merge these changes to. Right click on your trunk commit in the transactions pane and click merge. This will merge the changes from that revision into your working copy of the branch. After that, simply commit to the branch.
If you only want to merge one or two files from a given commit, you can do that too. The amount of time this saves is huge when you don't have to manually merge these changes across branches.

Conclusion

The capability to merge single revisions across branches is huge, but there are still many other features that separate SmartSVN from the pack. For more detailed information on SmartSVN, check out their site at: http://www.syntevo.com/smartsvn/index.html

Also, just for the record, I'm not affiliated with Syntevo or receiving any compensation for this post.

Test your Java Regular Expressions ( REGEX ) Online

2010-01-21T06:26:00.000-08:00

If you've ever worked with regular expressions using the java.util.regex package in Java, you know it can be a slow process to thoroughly test your regular expression. I recently found a website that allows you to test your regex using the java.util.regex package and actually provides detailed feedback.
Using fileformat.com's Regular Expression Test Page, you can run test data through Java's regex engine and see the Matcher's output methods such as matches(), replaceFirst(), replaceAll(), lookingAt(), and find(). It will also show you the groups your regular expression creates.

The next time you are writing some fancy regular expressions for use in the java.util.regex package, make sure you use this handy tool for testing your regex in Java.

Fixing Slow Internet Browsing when using Firefox on Mac OSX

2010-01-13T19:08:00.000-08:00

If you're using Firefox on your Mac and having issues with slow web browsing, it may be due to the fact that Firefox is attempting to do IPV6 DNS lookups. The main symptom for this problem is that every time you attempt to browse to a new page, the status in the bottom left corner says "Locating whatever website..." and it takes a few seconds before the page starts to load. However, once you get to the page, browsing speeds up.

The problem is that Firefox attempts to resolve IPV6 addresses first then attempts to resolve IPV4 addresses by default. If your ISP's DNS server does not support IPV6, this will cause a delay. Fortunately, the good people at Mozilla gave us a way to change this setting.

To fix this, you can disable IPV6 in Firefox by following the steps below:

Open Firefox, type about:config in the address bar and hit enter.
You'll receive a warning, click the button that says "I'll be careful, I promise!"
Find the preference named network.dns.disableIPV6 and double click it to set it to true.
Restart Firefox

Now your browsing should speed up because Firefox will now only attempt to resolve addresses in IPV6 format. For more information about this issue check out http://kb.mozillazine.org/Network.dns.disableIPv6.

Securing Grails: Salting Passwords using the ReflectionSaltSource

2010-01-11T17:14:00.001-08:00

I'm a Grails newbie, however I have been using Spring Security for quite some time. One of the features I really like about Spring Security is the capability to provide either a system wide salt or reflection salt source for more secure password hashing.

As I was getting started on a Grails project, I was impressed with how easily the Acegi Spring Security plugin integrated with my project. However, after adding a few users with the same password, I realized the passwords were not being salted with user properties because all of the hashes were the same.

In order to add a Spring ReflectionSaltSource to our Grails authenticationDao, we'll need to start by declaring the salt source in our resources.groovy file.

import org.springframework.security.providers.dao.salt.ReflectionSaltSource

beans = {

    saltSource( ReflectionSaltSource ) {
        userPropertyToUse = "username"
    }

}

The code above creates a saltSource bean that we can use for dependency injection throughout our Grails project. The next step we need to take is to inject our ReflectionSaltSource into the daoAuthenticationProvider. I chose to do this within the bootstrap.groovy file. Below is a sample of my BootStrap.groovy.

class BootStrap {

     def daoAuthenticationProvider
     def saltSource

     def init = { servletContext ->

          daoAuthenticationProvider.saltSource = saltSource

     }

     def destroy = {}
}

We have now setup Acegi authentication to use our ReflectionSaltSource using the username property. However, one more gotcha remains. If you generated the RegistrationController using the Acegi plugin, you'll need to change how it encodes passwords for new users. Otherwise, it will continue to encode passwords the old way and authentication will never succeed.

In the RegistrationController, you'll see where it is calling on the authenticateService.encodePassword method to encode new passwords. Simply change that call to use the daoAuthenticationProvider's passwordEncoder like so:

daoAuthenticationProvider.passwordEncoder.encodePassword( passwd, username )

Now all of your users' password hashes will be salted with their username. Keep in mind, that you can't allow users to change their usernames otherwise, they will not be able to login. This simple step makes your Grails application more secure as intruders will not easily be able to identify common password hashes if they get access to your database.

Optimizing Oracle Inserts with FORALL

2009-06-18T17:41:00.001-07:00

I was recently faced with a daunting task of processing approximately 300,000 records for an insert into a denormalized table in an Oracle 10g database for historical reporting. The table that was the starting point had no records that had been modified, however all of the tables the table was related to had many modifications on all records. The project required me to go through every record in the starting table and then reconstruct all of the related rows to their historical value at the time the base records were inserted.

My initial go around at the task involved writing stored procedures that retrieved historical values for a given primary key and date value. By digging through the historical values from audit tables, I was able to determine the value for the given row at the given date.

After writing the necessary stored procedures to grab historical values from the related tables, I wrote a PL/SQL block to loop through all of the base records and then grab the historical values into rowtype variables and performed the inserts. The initial run of the script worked perfectly however it took approximately 7 hours and 15 minutes to run averaging about 11.5 records per second.

While this run time was acceptable to the requirements, I knew there had to be a better way to accomplish the task with a faster run time. Knowing the benefits of Oracle's forall, I converted all of my PL/SQL stored procedures that retrieved historical values to views so I could boil the inserts down to one SQL statement. Then I converted my PL/SQL for loop to a forall with the one insert statement.

My new and improved script ran in 6 hours even and averaged 13.9 records per second, a saving of 20%. The next time you are looking to cut some time from a time consuming data insert, try out the forall query. You can learn more about forall at Oracle's website.

Writing flat files in Java with Flatworm

2009-04-04T13:44:00.001-07:00

I previously wrote about how to parse and read flat files in Java using Flatworm. In this post, I'll show you how you can write flat files with Flatworm. The benefit to using the Flatworm library for flat file creation is that you can use the exact same file descriptor document you use to parse a particular format. The developer doesn't have to concern themselves with the details of the file layout and can focus on the data going into the file.

Writing Using Our Previous Example

Let's revisit the sample data format we used when we explored how to parse and read flat files with Flatworm.

Name	Start	End	Length	Type
Type	1	2	2	Char
First	3	27	25	Char
Middle	28	52	25	Char
Last	53	77	25	Char
Acct. ID	78	92	15	Char

Below is the sample flat file we will be writing.

CDJOHN                     MARK                     DOE                      111111111111111
CDPAUL                     RICHARD                  STEPHENS                 222222222222222
CDRINGO                    JACK                     ERICSON                  333333333333333

Now that we know the file format of our flat file and we have some
sample data to parse we'll need to create an XML document describing
our file format for Flatworm.

<?xml version="1.0" encoding="ISO-8859-1"?>
<!--<!DOCTYPE file-format SYSTEM "http://www.blackbear.com/dtds/flatworm-data-description_1_0.dtd">-->
<file-format>
  <converter name="char" class="com.blackbear.flatworm.converters.CoreConverters" method="convertChar" return-type="java.lang.String"/>
  <record name="clientData">
      <record-ident>
          <field-ident field-start="0" field-length="2">
              <match-string>CD</match-string>
          </field-ident>
      </record-ident>
      <record-definition>
          <bean name="client" class="org.javaconfessions.sample.Client"/>
          <line>
              <record-element length="2"/>
              <record-element length="25" beanref="client.firstName" type="char">
                  <conversion-option name="justify" value="left"/>
                  <conversion-option name="pad-character" value=" "/>
              </record-element>
              <record-element length="25" beanref="client.middleName" type="char">
                  <conversion-option name="justify" value="left"/>
              </record-element>
              <record-element length="25" beanref="client.lastName" type="char">
                  <conversion-option name="justify" value="left"/>
              </record-element>
              <record-element length="15" beanref="client.accountId" type="char">
                  <conversion-option name="justify" value="left"/>
              </record-element>
          </line>
      </record-definition>
  </record>
</file-format>

Closer Look at the Descriptor File

Looking further at the XML descriptor file, you can see it is rather simple to describe our file format for Flatworm. The record tag is the beginning of describing our client data records. Within the record tag, we have our record-ident
tag. This is so Flatworm knows how to identify the types of records in
a flat file. Most flat file formats have different types of records
including header, footer, detail, batch headers, batch footers, etc.
This mechanism allows Flatworm to parse out all of these different
record types from the same file. The field-ident tag gives the specifics on how to identify the record. Field-start and field-length identifies what to test to identify the record type. Within the match-string tags
is where the text that would be used to identify this record as a
clientData record. In the descriptor above, we have described
clientData records as starting with the characters CD.

The next section of the record description is the record-definition tag.
This is where we actually map out each record element to a bean
property for Flatworm. This section of the document starts with a bean definition that tells Flatworm which Java class to use when parsing this record type. The record-element tags setup where each field in the record is located, the data type, and where to plug it into the Java bean during parsing.

Here is the source code for my Client bean.

package org.javaconfessions.sample;

public class Client {

 private String firstName;
 private String middleName;
 private String lastName;
 private String accountId;

 public String getFirstName() {
     return firstName;
 }

 public void setFirstName(String pFirstName) {
     firstName = pFirstName;
 }

 public String getMiddleName() {
     return middleName;
 }

 public void setMiddleName(String pMiddleName) {
     middleName = pMiddleName;
 }

 public String getLastName() {
     return lastName;
 }

 public void setLastName(String pLastName) {
     lastName = pLastName;
 }

 public String getAccountId() {
     return accountId;
 }

 public void setAccountId(String pAccountId) {
     accountId = pAccountId;
 }

 @Override
 public String toString() {
     return "First Name: " + firstName + "\nMiddleName: " + middleName
             + "\nLastName: " + lastName + "\nAccount ID: " + accountId
             + "\n";
 }

}

Now that we have described the data model for our project, below is a sample class that populates a couple of Client beans and then has Flatworm write out the data in our specified file format.

package org.javaconfessions.sample;

import com.blackbear.flatworm.FileCreator;

public class ClientDataWriter {

 public static void main(String[] args) {
     FileCreator fileCreator = null;
     try {
         fileCreator = new FileCreator(args[0], args[1]);
         fileCreator.open();
         fileCreator.setRecordSeperator("\n");
         Client client = new Client();
         fileCreator.setBean("client", client);
         client.setFirstName("JOHN");
         client.setMiddleName("MARK");
         client.setLastName("DOE");
         client.setAccountId("111111111111111");
         fileCreator.write("clientData");
         client.setFirstName("PAUL");
         client.setMiddleName("RICHARD");
         client.setLastName("STEPHENS");
         client.setAccountId("222222222222222");
         fileCreator.write("clientData");
         client.setFirstName("RINGO");
         client.setMiddleName("JACK");
         client.setLastName("ERICSON");
         client.setAccountId("333333333333333");
         fileCreator.write("clientData");
     } catch (Exception e) {
         e.printStackTrace();
     } finally {
         try{
             fileCreator.close();
         } catch( Exception e ) {
             e.printStackTrace();
         }
     }
 }
}

A Closer Look at ClientDataWriter.java

Looking at the sample code above, we can see that first we are creating a FileCreator object using parameters passed in. The first parameter given is the path to our Descriptor XML Document and the second parameter will be the location where the data file should be written. Next we call the open() method on the FileCreator object to open our file. The next call sets the record separator for our data file which in this case is a new line.

After getting our FileCreator object setup, we instantiate a Client bean. After instantiating our Client bean, we call setBean() on the FileCreator to tell the FileCreator about our Client bean and what type of bean it is. Notice that "client" is the name of our bean in the descriptor file. The next part is pretty self explanatory, we setup our first client record, then call fileCreator.write() passing in the type of record we want to write, in this case a "clientData" record. Then we repeat for the second and third records.

After writing out the records, we call the close() method on the FileCreator to close out the data file.

Gotcha

The Flatworm FileCreator will always put the record identifiers at the beginning of the record in the flat file. This doesn't always work because sometimes, you may have a situation where the record identifier in the middle of the record. I have a patch for the FileCreator below if you want to change this in the source. If you apply this patch, you will need to setup the record identifier portion of the bean description with the default values, in our case here "CD". However, you will have to do this for all constant data types in your file format that aren't specified in your bean or your file format will be incorrect.

Index: src/com/blackbear/flatworm/FileCreator.java
--- src/com/blackbear/flatworm/FileCreator.java Base (1.3)
+++ src/com/blackbear/flatworm/FileCreator.java Locally Modified (Based On 1.3)
@@ -212,12 +212,12 @@
// record-ident contain what is considered hard-coded data
// for the output line, these can be used to uniquely identify
// lines for parsers. We need to write them out.
- Vector recIdents = record.getFieldIdentMatchStrings();
+ /*Vector recIdents = record.getFieldIdentMatchStrings();
for (Iterator itRecIdents = recIdents.iterator(); itRecIdents.hasNext();)
{
String id = (String) itRecIdents.next();
bufOut.write(id + delimit);
- }
+ }*/

// Iterate over record-element items
Vector recElements = line.getRecordElements();

How to read and parse flat files in Java

2009-02-21T14:52:00.001-08:00

Parsing files and their formats can be pretty painful no matter what programming language you are using. One open source project, Flatworm, looks to make reading, parsing, and writing files, much easier in Java. You define the file format in XML and Flatworm will break out the records into Java beans for you. You can read large files, file formats that have multiple-line records, and any other flat file format in existence today with this Java API.

Simple Example

Let's look at a simple example where we need to parse out a flat file into Java objects for processing. In our example we will need to parse client data using the file format below.

Name	Start	End	Length	Type
Type	1	2	2	Char
First	3	27	25	Char
Middle	28	52	25	Char
Last	53	77	25	Char
Acct. ID	78	92	15	Char

Below is the sample flat file we will be parsing.

CDJOHN                     MARK                     DOE                      111111111111111
CDPAUL                     RICHARD                  STEPHENS                 222222222222222
CDRINGO                    JACK                     ERICSON                  333333333333333

Now that we know the file format of our flat file and we have some sample data to parse we'll need to create an XML document describing our file format for Flatworm.

<?xml version="1.0" encoding="ISO-8859-1"?>
<!DOCTYPE file-format SYSTEM "http://www.blackbear.com/dtds/flatworm-data-description_1_0.dtd">
<file-format>
<converter name="char" class="com.blackbear.flatworm.converters.CoreConverters" method="convertChar" return-type="java.lang.String"/>
<record name="clientData">
<record-ident>
<field-ident field-start="0" field-length="2">
<match-string>CD</match-string>
</field-ident>
</record-ident>
<record-definition>
<bean name="client" class="org.javaconfessions.sample.Client"/>
<line>
<record-element length="2"/>
<record-element length="25" beanref="client.firstName" type="char">
<conversion-option name="justify" value="left"/>
</record-element>
<record-element length="25" beanref="client.middleName" type="char">
<conversion-option name="justify" value="left"/>
</record-element>
<record-element length="25" beanref="client.lastName" type="char">
<conversion-option name="justify" value="left"/>
</record-element>
<record-element length="15" beanref="client.accountId" type="char">
<conversion-option name="justify" value="left"/>
</record-element>
</line>
</record-definition>
</record>
</file-format>

Closer Look at the Descriptor File

<file-format> - This tag is required and serves as the root node of our descriptor file.

<converter> - This tag is used to declare new converters to be used by the flatworm parser.

Looking further at the XML descriptor file, you can see it is rather simple to describe our file format for Flatworm. The record tag is the beginning of describing our client data records. Within the record tag, we have our record-ident tag. This is so Flatworm knows how to identify the types of records in a flat file. Most flat file formats have different types of records including header, footer, detail, batch headers, batch footers, etc. This mechanism allows Flatworm to parse out all of these different record types from the same file. The field-ident tag gives the specifics on how to identify the record. Field-start and field-length identifies what to test to identify the record type. Within the match-string tags is where the text that would be used to identify this record as a clientData record. In the descriptor above, we have described clientData records as starting with the characters CD.

The next section of the record description is the record-definition tag. This is where we actually map out each record element to a bean property for Flatworm. This section of the document starts with a bean definition that tells Flatworm which Java class to use when parsing this record type. The record-element tags setup where each field in the record is located, the data type, and where to plug it into the Java bean during parsing.

Here is the source code for my Client bean.

package org.javaconfessions.sample;

public class Client {

private String firstName;
private String middleName;
private String lastName;
private String accountId;

public String getFirstName() {
return firstName;
}

public void setFirstName(String pFirstName) {
firstName = pFirstName;
}

public String getMiddleName() {
return middleName;
}

public void setMiddleName(String pMiddleName) {
middleName = pMiddleName;
}

public String getLastName() {
return lastName;
}

public void setLastName(String pLastName) {
lastName = pLastName;
}

public String getAccountId() {
return accountId;
}

public void setAccountId(String pAccountId) {
accountId = pAccountId;
}

public String toString() {
return "First Name: " + firstName + "\nMiddleName: " + middleName
+ "\nLastName: " + lastName + "\nAccount ID: " + accountId
+ "\n";
}

}

Now that we have all of our data model code setup, the step is to write the code that will be responsible for populating our Java bean with the parsed data. I have posted a simple parsing Class below that will parse the flat file using Flatworm, and print each item out to the console.

package org.javaconfessions.sample;

import com.blackbear.flatworm.ConfigurationReader;
import com.blackbear.flatworm.FileFormat;
import com.blackbear.flatworm.MatchedRecord;
import java.io.BufferedReader;
import java.io.FileInputStream;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.util.logging.Level;
import java.util.logging.Logger;

public class ClientDataParser {

public static void main(String[] args) {
ConfigurationReader parser = new ConfigurationReader();
try {
FileFormat ff = parser.loadConfigurationFile(args[0]);
InputStream in = new FileInputStream( args[1] );
BufferedReader bufIn = new BufferedReader( new InputStreamReader( in ) );
MatchedRecord results;
while( ( results = ff.getNextRecord(bufIn)) != null ) {
System.out.println( results.getBean("client") );
}
} catch (Exception ex) {
Logger.getLogger(ClientDataParser.class.getName()).log(Level.SEVERE, null, ex);
}

}

}

Now after compiling, we just need to execute java ClientDataParser /path/to/format.xml /path/to/datafile.txt and you should get the following output:

First Name: JOHN
MiddleName: MARK
LastName: DOE
Account ID: 111111111111111

First Name: PAUL
MiddleName: RICHARD
LastName: STEPHENS
Account ID: 222222222222222

First Name: RINGO
MiddleName: JACK
LastName: ERICSON
Account ID: 333333333333333

Summary
Now you have an idea of how Flatworm works to simplify parsing flat files. Next learn how to write flat files using Flatworm.

Switching from Windows to a Macbook

2009-02-04T18:46:00.001-08:00

I have never owned a Mac before and recently made the plunge into the Mac world. I recently purchased the new aluminum case macbook and I have to say as a long time Windows user it is an amazing device. However, there are some differences that take some used to getting used to and I wanted to help out others who may be diving into the Mac world for the first time.

Common Keyboard Shortcuts
The first thing you'll notice on your new macbook is that the keyboard is different and therefore some of your well known Windows keyboard shortcuts are useless. For most common functions, you can simply replace the Ctrl with the Command key. The table below gives a simple illustration.

Function	Windows	Mac
Copy	Ctrl-C	Command-C
Cut	Ctrl-X	Command-X
Paste	Ctrl-V	Command-V

Not So Obvious Keyboard Operations
You'll notice that there are some keys missing on your new macbook. When you started using it, you may have said, "Why does my backspace say delete, and by the way, where is the delete key?" Well, this table answers those questions.

Function	Windows	Mac
Move to the Next Page (Page Down)	Page-Dn	Fn-Down Arrow
Move to the Previous Page (Page Up)	Page-Up	Fn-Up Arrow
Move to the End of the current line	End	Command-Right Arrow
Move to the Beginning of the Current Line	Home	Command-Left Arrow
Move to the End of a Document	Cntrl-End	Command-Down Arrow
Move to the Beginning of a Document	Cntrl-Home	Command-Up Arrow
Delete Text to the right of the cursor.	Delete	Fn-Delete
Find Next	F3	Command-G
Switch Running Applications	Alt-Tab	Command-Tab

Publishing to Blogger with Google Docs

2008-12-22T20:11:00.001-08:00

When I first started publishing to Blogger, I was extremely disappointed with the WYSIWG editor. Simple tasks such as inserting tables or images take a lot of time and you have to usually wind up writing HTML code to do it right. I have been looking for a better solution to do my blogging.

Google Docs
I recently took Google Docs out for a test drive and was impressed with how easy it was to write rich text documents through a web page. You can easily make format changes, build tables, and insert images. Google Docs also allows you to create presentations and spreadsheets.

Posting to Blogger with Google Docs
While working on a test document, I noticed there is a share button on the top right hand corner of the page. When you click the Share button, there are a couple of options available. If you select Publish as Web Page, you will presented with an option to Post to Blog. There will also be a link there so that you can setup your blog account settings. Once your Blogger account settings are saved in Google Docs, you can now easily post documents from Google Docs to your Blogger blog. Give it a try, I have been using this setup for a few weeks now and I have to say that it is the best way I have found to create Blogger posts.

Meld: Open Source diff and merge tool for Linux

2008-12-22T18:45:00.001-08:00

When I chose to move over to Linux from Windows on my development computer, I had to find an alternative diff and merge tool as I had been using TortoiseSVN for this functionality. That's when I found about Meld and I have been pleasantly surprised by this simple yet powerful diff tool.

Easy Installation

Since Meld is in the Ubuntu package archives, you can easily install it by opening Applications > Add/Remove. When Add/Remove opens, search for Meld and select to install Meld Diff Viewer. From the command line, you can also install Meld through apt with the following command: sudo apt-get install meld

In Place Editing

The main feature that has blown me away with Meld is the capability to edit files while you are comparing them. Not only can you do normal merge operations with files you are comparing, but you can actually edit the files as though they are in a text editor and Meld does realtime comparisons as you do this. This really comes in handy in all sorts of situations.

You can see in the image that while comparing the two files, I decided to throw in some extra text into file2 and it immediately updated the comparison. Of course if I want to move the change over to file1 all I have to do is click the black arrow and it will update file1. This is a very powerful feature and really saves time when you need to be able to make changes in a hurry to multiple files.

Easy Integration with RapidSVN

If you miss the file comparison feature found in TortoiseSVN under Windows, you can setup Meld to be your file comparison tool under RapidSVN in Linux. In RapidSVN, click on View from the menu bar and select Preferences. Once the Preferences Menu opens, click on the Diff Tool tab, type meld in the command text box and click OK. Now when you choose to diff files from within RapidSVN, it will use Meld to compare files.

Integration with Nautilus

For more siplicity, you can setup Nautilus so that Meld shows up in the right-click menu when you select two or more files. In order to do this, you'll need to install the Nautilus Actions Configuration application. Open up Applications > Add/Remove. When Add/Remove opens search for Nautilus Actions Configuration and install it. Once installed, go to System > Preferences > Nautilus Actions Configuration. Once it loads, click the Add button.

Set the Label to Meld. This is the text that will show up when you right click.

Set the Icon to wherever your Meld icon image file is. Mine is located at /usr/share/pixmaps/meld.png.

Set the path to meld

Set the parameters to %M. If you look at the legend, this will pass in a space-separated list of the selected files/folders with their full paths.

Click the Conditions tab.
Set Filenames to * and check the Match Case check box.
Set Mimetypes to */*
Select the Both radio button and check the Appears if selection has multiple files or folders check box under "Appears if selection contains."
Click OK

Now when you select two files in Nautilus and right click, you will have a menu item called Meld that will allow you to compare the files. Nice...

Pandora for the iPhone

2008-12-18T16:31:00.001-08:00

I have been using another awesome app for the iPhone which is Pandora. It is just like the Pandora Internet Radio in that you type in a band or song name and it builds a radio station around your entry. You can add radio stations to the app by adding band names and once you have more than one band name, you can do a quick mix which is essentially a mesh of all of the radio stations you have built. This is a really nice app that makes owning the iPhone even better. I listen to this while I'm at work, at the gym, or even while I'm just cleaning the house. It also includes the capability to rate songs that are played so that it can fine tune the types of music you will enjoy. As the iPhone is the first Apple product I have ever owned, I am definitely not a fanboy, but I may be soon.

How to search for a percent (%) sign in a like query

2008-12-18T13:04:00.001-08:00

The percent (%) sign in a SQL LIKE query is used as a wildcard. If you need to look for text containing a percent (%) sign, then you'll have to use an escape character to specify that the SQL database should look for an actual percent sign. The query below is an example of how to do this:

SELECT * FROM my_table WHERE some_column LIKE '%\%%' ESCAPE '\';

The query above defines the backslash as an escape character so that it will search for any row that has a percent sign in some_column.

Generate Charts and Graphs in Java Using JFreeChart

2008-12-17T00:04:00.000-08:00

I was recently looking for an open source charting and graphing API for Java when I came across JFreeChart. They describe JFreeChart as a free 100% Java chart library that makes it easy for developers to display professional quality charts in their applications.

One of the first things I noticed was that there is very little to no documentation on how to use JFreeChart and I am assuming that is because they sell their Developer's Guide to support the project. While that is definitely a good way to support their project, it makes it hard for people to test out the API. So, hopefully, I can help you get started with using JFreeChart. I strongly encourage purchasing the Developer's Guide if you plan on using JFreeChart in your projects.

A Simple Line Chart
We'll take a look at a simple dataset that would be used in a basic line graph using JFreeChart. Our datasets will represent the number of visitors counted at local parks over a 7 day period.

Park 1
Day	Number of Visitors
1	10
2	4
3	6
4	12
5	11
6	39
7	33

Park 2
Day	Number of Visitors
1	23
2	15
3	18
4	5
5	52
6	66
7	83

We are going to create a graph using the dataset above with the JFreeChart API. The sample class below will show how to generate a line graph using the data above.


package org.javaconfessions.sample;
import org.jfree.chart.ChartFactory;
import org.jfree.chart.JFreeChart;
import org.jfree.chart.plot.PlotOrientation;
import org.jfree.data.xy.XYSeries;
import org.jfree.data.xy.XYSeriesCollection;
import org.jfree.chart.ChartUtilities;

import java.io.File;
import java.io.FileOutputStream;

class XYChartSample {

public void buildChart() throws Exception {

   XYSeriesCollection dataset = new XYSeriesCollection();
   XYSeries xySeries = new XYSeries( "Park 1" );
   xySeries.add( 1, 10 );
   xySeries.add( 2, 4 );
   xySeries.add( 3, 6 );
   xySeries.add( 4, 12 );
   xySeries.add( 5, 11 );
   xySeries.add( 6, 29 );
   xySeries.add( 7, 33 );

   XYSeries xySeries2 = new XYSeries( "Park 2" );
   xySeries2.add( 1, 23 );
   xySeries2.add( 2, 15 );
   xySeries2.add( 3, 18 );
   xySeries2.add( 4, 5 );
   xySeries2.add( 5, 52 );
   xySeries2.add( 6, 66 );
   xySeries2.add( 7, 83 );

   dataset.addSeries( xySeries );
   dataset.addSeries( xySeries2 );

   JFreeChart chart = ChartFactory.createXYLineChart( "Park Chart",
                                                      "Day",
                                                      "# of Visitors",
                                                      dataset,
                                                      PlotOrientation.VERTICAL,
                                                      true,
                                                      false,
                                                      false );

   FileOutputStream out = new FileOutputStream( new File( "parkchart.png" ) );
   ChartUtilities.writeChartAsPNG( out, chart, 400, 200 );

}

}

This is just a basic example of how you can generate charts using JFreeChart. You can see at the end I also show an example of another useful feature found in JFreeChart, the capability to write out charts as images. In this example, I wrote out the chart as a PNG file. Here is an example of how that chart would look if you ran the code above.

Better Performance with Oracle SQL Developer in Ubuntu Linux

2008-10-28T09:11:00.000-07:00

Having switched to Ubuntu recently on my work machine I have been hooking up to our databases using Oracle's SQL Developer. One major issue that I have had is that sometimes SQL Developer would get hung while attempting to the databases. The connecting status dialog would pop up, but it would never actually connect.

It would usually be solved by restarting SQL Developer a few times, but it got to be rather annoying. I decided to devote a little time to trying to find a solution for this little annoyance and I believe I may have cured my SQL Developer connection woes by changing to the OCI/Thick JDBC Driver.

In SQL Developer go to Tools > Preferences to open the Preferences menu.
Expand the Database section and click Advanced Parameters.
Then check the box labeled "Use OCI/Thick driver".
Restart SQL Developer

Invalid Mount Options Error When Mounting External Hard Drive in Ubuntu Linux

2008-10-11T04:33:00.000-07:00

Being a recent convert to Ubuntu Linux, I ran into a problem that others may have experienced. When I plugged in my usb external hard drive I received an Invalid Mount Options error. I found a lot of information on the forums, but nothing was working. However, I discovered that Ubuntu was basically attempting to mount my FAT32 external hard drive as a CD-ROM. In order to fix this problem you need to edit /etc/fstab and comment out the line below by putting a # at the beginning of the line:

/dev/sdb1       /media/cdrom0   udf,iso9660 user,noauto,exec,utf8 0       0

The line should look like this after the edit:

#/dev/sdb1       /media/cdrom0   udf,iso9660 user,noauto,exec,utf8 0       0

After this change, voila, the drive mounts as desired.

Convert Between Base 10, Base 62, Base 36, Base 16, Base 8, Base 2 in Java

2008-09-28T10:16:00.000-07:00

I've already written how you can Convert Between Base 10 and Base 62 in PL/SQL, and in this post I'll show you how to convert between base 10 and base 62 in Java. Well, actually you can pretty much convert between anything you want using the Java utility class below. Enjoy!


package org.javaconfessions.baseconverter;


public class BaseConverterUtil {

    private static final String baseDigits = "0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz";

    public static String toBase62( int decimalNumber ) {
        return fromDecimalToOtherBase( 62, decimalNumber );
    }

    public static String toBase36( int decimalNumber ) {
        return fromDecimalToOtherBase( 36, decimalNumber );
    }

    public static String toBase16( int decimalNumber ) {
        return fromDecimalToOtherBase( 16, decimalNumber );
    }

    public static String toBase8( int decimalNumber ) {
        return fromDecimalToOtherBase( 8, decimalNumber );
    }

    public static String toBase2( int decimalNumber ) {
        return fromDecimalToOtherBase( 2, decimalNumber );
    }

    public static int fromBase62( String base62Number ) {
        return fromOtherBaseToDecimal( 62, base62Number );
    }

    public static int fromBase36( String base36Number ) {
        return fromOtherBaseToDecimal( 36, base36Number );
    }

    public static int fromBase16( String base16Number ) {
        return fromOtherBaseToDecimal( 16, base16Number );
    }

    public static int fromBase8( String base8Number ) {
        return fromOtherBaseToDecimal( 8, base8Number );
    }

    public static int fromBase2( String base2Number ) {
        return fromOtherBaseToDecimal( 2, base2Number );
    }

    private static String fromDecimalToOtherBase ( int base, int decimalNumber ) {
        String tempVal = decimalNumber == 0 ? "0" : "";
        int mod = 0;

        while( decimalNumber != 0 ) {
            mod = decimalNumber % base;
            tempVal = baseDigits.substring( mod, mod + 1 ) + tempVal;
            decimalNumber = decimalNumber / base;
        }

        return tempVal;
    }

    private static int fromOtherBaseToDecimal( int base, String number ) {
        int iterator = number.length();
        int returnValue = 0;
        int multiplier = 1;

        while( iterator > 0 ) {
            returnValue = returnValue + ( baseDigits.indexOf( number.substring( iterator - 1, iterator ) ) * multiplier );
            multiplier = multiplier * base;
            --iterator;
        }
        return returnValue;
    }

}

Part Three - SSH Channels

2008-09-27T08:20:00.001-07:00

We need to understand the SSH channel process in order to continue, so what are channels? All terminal sessions, forwarded connections, etc. are channels. Either side may open a channel and multiple channels are multiplexed into a single connection. Channels are flow-controlled, that is no data may be sent to a channel until a message is received to indicate that window space is available.

Channels can be whatever you want them to be, there are many channel types already defined within the SSH protocol specification, such as the 'session' or 'tcpip-forward' channels. These are the mechanism that provide the transport for your session and port forwarding data.

When you want to open a channel you send a request to the SSH server, the server will then either respond to that request with a confirmation or failure message. Your channel is identified by a name, for instance lets say we want to create a channel to echo information back to the client, we would call the channel "echo@3sp.com";. You should always name your channel using the name@domain syntax.

J2SSH provides a mechanism for developing custom channels, the com.sshtools.j2ssh.connection package contains a number of classes to help you. First the Channel class is the abstract base for all channels. Lets take a look at its abstract methods, the first method is self explainitory, it returns the name/type of the channel.

public String getChannelType() {
return "echo@3sp.com";
}

The next two methods provide the settings for the flow control. The values returned by the following mehtods set the boundary for the window space. The channel will never provide more window space than the maximum and will increase window space automatically (back up to the maximum) when the minimum is reached.

protected int getMinimumWindowSpace() {
return 1024;
}
protected int getMaximumWindowSpace() {
return 65535;
}

The maximum packet size setting is the maximum amount of data that the remote side can send in one single packet. This value should not exceed the maximum window space.

protected int getMaximumPacketSize() {
return 32768;
}

When a request is made to open a channel, the client can send some data with the open request so that the server may process the request based on additional information. Your channel should return this information (if any) in the getChannelOpenData method, if there is no data simply return null.

public byte[] getChannelOpenData() {
return null;
}

When the server confirms that the channel is open it can also provide data in response to the channel open information. It does so by returning this information in the getChannelConfirmationData method. Again if no data is required to be sent to the remote side simply return null.

public byte[] getChannelConfirmationData() {
return null;
}

Once the channel has been opened, the channel mechanism calls the channels onChannelOpen method. You should not perform any expensive processing in this method since it will lock up the protocol and you will not be able to send data.

protected void onChannelOpen() throws java.io.IOException {
/** Your channel is open so do stuff if you want **/
}

Once the channel is open you can send data using the channels sendChannelData method, this type of data is received by the channel with the following method.

protected void onChannelData(SshMsgChannelData msg) throws java.io.IOException {
// Channel data has arrived, process it!
}

There is also an extended data channel which can be used, the extended data has a type field which can be used to identify different application defined data types. This is received with the onChannelExtData method and sent with sendChannelExtData.

protected void onChannelExtData(SshMsgChannelExtendedData msg) throws java.io.IOException {
/**@todo Implement this com.sshtools.j2ssh.connection.Channel abstract method*/
}

There is also a request mechanism seperate to the channel data which can be used. This provides named based requests. For example in our echo channel we could have a request to turn echo on and off. Requests are recieved with the onChannelRequet method and sent with sendChannelRequest.

protected void onChannelRequest(String requestname, boolean wantreply, byte[] requestdata)
throws java.io.IOException {
/**@todo Implement this com.sshtools.j2ssh.connection.Channel abstract method*/
}

When you no longer wish to send data you can set the local side to EOF by using setLocalEOF. When the remote server sends EOF the onChannelEOF method is called.

protected void onChannelEOF() throws java.io.IOException {
/**@todo Implement this com.sshtools.j2ssh.connection.Channel abstract method*/
}

Finally, to close the channel you can use the close method. When the channel is closed by either side the onChannelClose method is called.

protected void onChannelClose() throws java.io.IOException {
/**@todo Implement this com.sshtools.j2ssh.connection.Channel abstract method*/
}

Ok so now we know the basic structure how do we implement our channel and use it from either side of the connection? Heres our server side channel implelentation. This simply returns any data sent to it if echo is on.

public class EchoChannel extends Channel {

boolean echo = true;

public EchoChannel() {

}

public String getChannelType() {
 return "echo@3sp.com";;
}

protected void onChannelRequest(String requestname, boolean wantreply, byte[]
              requestdata) throws java.io.IOException {
  if(requestname.equals("echo-off@3sp.com";)) {
    echo = false;
    if(wantreply)
       connection.sendChannelRequestSuccess(this);
    return;
   } else if(requestname.equals("echo-on@3sp.com";)) {
    echo = true;
    if(wantreply)
      connection.sendChannelRequestSuccess(this);
    return;
  }

  if(wantreply)
    connection.sendChannelRequestFailure(this);
}

protected void onChannelExtData(SshMsgChannelExtendedData msg) throws java.io.IOException {

}

protected void onChannelData(SshMsgChannelData msg) throws java.io.IOException {
 if(echo)
  sendChannelData(msg.getChannelData());
}

protected int getMaximumPacketSize() {
return 32768;
}

protected void onChannelEOF() throws java.io.IOException {

}

protected void onChannelClose() throws java.io.IOException {

}

public byte[] getChannelOpenData() {
return null;
}

protected int getMinimumWindowSpace() {
return 1024;
}

protected int getMaximumWindowSpace() {
return 65535;
}

protected void onChannelOpen() throws java.io.IOException {

}

public byte[] getChannelConfirmationData() {
return null;
}
}

Now we need to configure the server to accept the channel and create an instance of our EchoChannel when a request is made. To do this we will need to create a ChannelFactory that can create the channel, this is a simple interface

public interface ChannelFactory {


public Channel createChannel(String channelType, byte[] requestData)
    throws InvalidChannelException;
}

So lets create an EchoChannelFactory implementation

public class EchoChannelFactory implements ChannelFactory {

 public Channel createChannel(String channelType, byte[] requestData)
       throws InvalidChannelException {
  
     if(channelType.equals("echo@3sp.com";)) {
       return new EchoChannel();
     }
   
     throw new InvalidChannelException("Only echo channels allowed by this factory");
 }
}

So now were ready to configure the server. In my previous articles you will remember we implemented the configureServices method of the SshServer? To allow this channel to be opened, we simply add the following line to the configureServices implementation

connection.addChannelFactory("echo@3sp.com";, new EchoChannelFactory());

Your server should now be configured to support your channel. So lets look at how to invoke the channel from the client.

First we need to create a client side implementation of the channel, but we need it to be simpler with a set of IOStreams perhaps??? Well the hard work is already done for you, take a look at the IOChannel class. This provides an InputStream and OutputStream for the channels data. Heres the implementation, it has less methods to implement since the channel data is now handled by the parent class.

public class EchoChannelClient extends IOChannel {

public EchoChannelClient() {

}

public String getChannelType() {
  return "echo@3sp.com";;
}

protected void onChannelRequest(String requestname,
                                 boolean wantreply,
          byte[] data) throws java.io.IOException {
  if(wantreply)
    connection.sendChannelRequestFailure(this);
}

protected int getMaximumPacketSize() {
 return 32768;
}

public byte[] getChannelOpenData() {
  return null;
}

protected int getMinimumWindowSpace() {
  return 1024;
}

protected void onChannelOpen() throws java.io.IOException {

}

protected int getMaximumWindowSpace() {
  return 65535;
}

public byte[] getChannelConfirmationData() {
  return null;
}
}

We need to add a method for turning the echo on and off.

public void setEcho(boolean echo) throws java.io.IOException {
if(echo)
  connection.sendChannelRequest(this, "echo-on@3sp.com";, false, null);
else
  connection.sendChannelRequest(this, "echo-off@3sp.com";, false, null);
}

Now were ready to go since any data we write to the channels outputstream, which we obtain by using getOutputStream will be returned to the InputStream if echo is on.

To use the channel using an SshClient instance simply

EchoChannelClient echo = new EchoChannelClient();

if(ssh.openChannel(echo)) {
// Channel is open
echo.getOutputStream().write("hello world!".getBytes());

// Read it back from the inputstream
byte[] buf = new byte[32];
echo.getInputStream().read(buf);
}

So of course your requirements are probably much more complex, if you want the server to handle IOStreams you can use the IOChannel instead of the Channel, in the end its up to you....... with a little imagination ;-)

Part Two - Building an SSH server to provide your applications services.

2008-09-27T08:18:00.001-07:00

In the previous part of this article I showed you how to secure your client-application server by using J2SSH to create an embeded tunnel to an SSH server installed on the same host as your server. Whilst the tunnel is secure it does not provide access to the strong public key authentication that SSH provides. In this article we take no prisoners, were going to implement an SSH server and transfer your existing application server functionality into the SSH server itself so that your application can take advantage of public key authentication and other SSH services such as SFTP.

Starting the basic framework server
First things first, even before you think about how to get your existing application functionality into the SSH server, you need to get the server itself running and accepting each connection and authenticating the user. Make sure you have the source version of J2SSH since we're going copy the com.sshtools.daemon.SshDaemon source code to create your server.

If you examine this class you will see that it has a main method and so we're going to use this class to configure, start and stop your server.Lets not worry about what the SshDaemon is doing right now, lets just make sure we can start and stop the server. You may have noticed that the main method allows a couple of command line arguments -start and -stop. No guessing for what these do! Before we start the server there are a couple of configuration items we need to attend to. First of all the server needs a public key to identify itself to clients. Generate a key pair using the J2SSH command line utility ssh-keygen.

¢ ssh-keygen.bat -t dsa -b 1024 server_host_key

When prompted to enter a passprase simply press return and confirm that you do not want to encrypt the private key, server host keys cannot be encrypted since there is no where to store the passphrase!

Copy the host key into the J2SSH conf directory, whilst you're in there open up the server configuration file server.xml and I'll take you over some of the other options. Make sure that there is a element whose PrivateKeyFile attribute is pointing to the private key file you have just generated.

Next make sure that the element is set to the port you wish your server to bind, if you have another SSH server running you might want to change this to a non standard port for now.

Finally the element is the local port which the server listens for commands, there is only one command currently implemented and that is stop.

You should now be able to start the server by executing the SshDaemon class, you will need to set the sshtools.home System property to point to the root installation directory of J2SSH. This will be used to find the XMl configuration files. Once you have the server running (remember to pass in the -start option on the command line) it should accept connections, but if you connect and try to authenticate it will fail because there is no authentication provider installed. So that's our next step.

Creating an authentication provider

The authentication provider allows you to define how users are authenticated when they connect to the SSH server. Your application may already have a mechanism for this, perhaps by searching a database table, or an encrypted password file. In any case you will want to transfer this process to the authentication provider, since I cannot help you on that front, I will show you how to implement a basic mechanism and explain the methods you are required to provide. To get things moving perhaps you should just implement this with me to see how the process fits together.

The abstract NativeAuthenticationProvider is where we start, we are going to extend this class to implement an authentication provider that examines the user name and logs them into the system. For simplicity we will hard code this in this example.

class MyAuthenticationProvider extends NativeAuthenticationProvider {

}

Now lets look at the methods we need to implement, first there is getHomeDirectory. This should return the home directory of the user whose username is passed as a parameter into the method. This will be used for public key configuration and also servers as the default directory for the SFTP server that the framework will provide.

public String getHomeDirectory(String username)
       throws IOException {
return "/home/users/martianx";
}

There are two logonUser methods that we need to implement next, the first provides a username and password which should be checked. If the account exists and the password is correct you should perform any processing that would log the user into the system and return true to indicate that the logon was a success. Optionally you can throw a password change exception if the users password has expired (more about that later).

public boolean logonUser(String username,
                      String password)
  throws PasswordChangeException,
         IOException {

return username.equals("martianx")
         && password.equals("foo"));

}

The second logonUser method only provides the username. This is called when a user log's into the system using public key authentication. When the method is called the server has already verified the identity of the user through their public key so you should automatically log the user in without restriction.

public boolean logonUser(String username) throws IOException {

return true; // This is for public key authentication and will only
            // be called once a key has been verified
 
}

The logoff User method simply should log the user out of the system, performing any necersary processing that your application server requires. In our example we will do nothing

public void logoffUser() throws IOException {
// do nothing yet
}

Finally the changePassword method is called if you had previously had thrown a PasswordChangeException from the logonUser method. This method should change the password and return its success (or failure). Our example will return false since we do not support password change yet.

public boolean changePassword(String username,
                           String oldpassword,
                           String newpassword) {
return false;   // We dont support this yet
}

So that's the authentication provider implemented, you should have a good idea about how to go about integrating your servers authentication mechanism now? But first we need to have a look at this provider in action, so the next step is to configure the server to load the provider at startup.

The platform configuration file

The providers are configured in a platform configuration file. The actual name of the file should be set to your application's name, but an example is provided in the J2SSH distribution and its called platform.xml. Lets copy the file and create myplatform.xml and configure it for our new authentication provider.

In the file locate the element and enter the FQN of the authentication implementation we just created.

com.myplatform.MyAuthenticationProvider

Save the file and before we start the server we need to make sure that the server will load the provider by setting a VM parameter, use

-Dsshtools.platform=myplatform.xml

Now start the server and try to connect using a standard SSH client and the authentication information we set in our example. The connection will still fail but you should get prompted for autehntication information and see an error message, for example using Putty as a client will probably show a pty refused message. This is actually a success since the pty is only requested once authentication has succeeded.

The next step is to remove this error message and were going to acheive that by creating a NativeProcessProvider.

Creating a NavtiveProcessProvider

The NativeProcessProvider allows the server to provide shell access and other session based services such as command execution. But you don't want a terminal I hear you say? Well that's fine but since your using the SSH framework your going to have to provide one, and you never know what you might be able to do in the future with a little imagination. For example one of the things I like to do is to provide terminal access to user accounts for self serve account maintenance, allowing them to login using a standard SSH client and change their password account details. It's always a good idea to allow some form of access into your application server that any standard client can access. What if your out of the office and really need to log in to change something, but don't have your laptop, connect using the SSHTerm applet and your connected again!

So what we are going to implement today is a basic terminal that outputs the message "This server does not provide shell access" and then disconnects. Let's start by declaring our class which extends NativeProcessProvider


public class UnsupportedShellProcess extends NativeProcessProvider {
private static String message = "This server does not provide shell access";
}

An instance of this class will be created for each session that is created, the process is required to provide the standard set of IO streams stdin, stdout and stderr. We will simply use a dynamic buffer class that J2SSH (which is similar to Piped streams) that will allow us to output something from the other end of the stream which will be read by the connected client.


import com.sshtools.j2ssh.io.DynamicBuffer;


DynamicBuffer stdin = new DynamicBuffer();
DynamicBuffer stderr = new DynamicBuffer();
DynamicBuffer stdout = new DynamicBuffer();

public InputStream getInputStream() throws IOException {
return stdin.getInputStream();
}

public OutputStream getOutputStream() throws IOException {
return stdout.getOutputStream();
}

public InputStream getStderrInputStream() {
return stderr.getInputStream();
}

When a command or shell is executed, the framework first requests that your provider create the process by calling the createProcess method, this method should initialize your provider with command and environment provided BUT NOT start the command. The framework will then start it sepeartley with a call to start. Since were writing a basic implementation we dont need to setup the process so were ust going to return true to indicate that the command is reading to start.

public boolean createProcess(String command,
                          Map environment)
      throws IOException {

// Of course this could be used to provide a wealth of features
// according to the executed command and the environment
return true;

}

Now when the process is started we output our message

public void start() throws IOException {
stdin.getOutputStream().write(message.getBytes());
}

Next the process is required to provide a default terminal provider, this is used in place of a command when the user requests to start a shell and so will be the command argument when createProcess is called, were not going to read this so just supply any string.

public String getDefaultTerminalProvider() {
return "UnsupportShell";
}

The client may request a pseudo terminal, again this is for a more advanced implementation than we need so in order to make sure that no client bombs out because we don't allocate one, we implement these methods to show we support and have allocated a terminal.

public boolean supportsPseudoTerminal(String term) {
 return true;
}

public boolean allocatePseudoTerminal(String term,
                                   int cols,
                                   int rows,
                                   int width,
                                   int height,
                                   String modes) {
return true;
}

The kill method should end the process, we will simply close all the buffers

public void kill() {
try {
 stdin.close();
}
catch (IOException ex) {
}
try {
 stdout.close();
}
catch (Exception ex1) {
}
try {
 stderr.close();
}
catch (Exception ex2) {
}
}

We need to provide the server with an evaluation on the process to determine whether its still active. We will return active whilst the whole message has not been read

public boolean stillActive() {
try {
 return stdin.getInputStream().available() > 0;
}
catch (IOException ex) {
 return false;
}
}

Finally this method should wait for the process to complete and then return the exit code. We are already planning on deprecating this method to provide a callback method but since its still here we need to implemenet it even if it is rather a crude implementation. We will simple wait for the message to be read and then return.

public int waitForExitCode() {
try {
 while (stdin.getInputStream().available() > 0) {
   try {
     Thread.sleep(1000); // Crude but necersary
   }
   catch (InterruptedException ex) {

   }
 }
}
catch (IOException ex1) {

}

return 0;
}

As you can see quite an easy implementation, when a connection is made that requests a shell or to execute a command the command will simply write out the message and once the message has been read the process will close.

To configure this in the platform configuration file you should set the element to point to your class implementation. If you run your server now and connect, you should no longer see the pty error message but instead should see the following text followed by a disconnect.

"This server does not provide shell access"

Now you should have a working SSH server which also provides SFTP access to your users. In the final part of this article I will show you how to implement an SSH channel and configure your client and server so that they directly talk to each other through SSH.

Part 1 - Securing your client-server application with J2SSH

2008-09-27T08:17:00.001-07:00

Using J2SSH is a great way to secure and enhance your insecure client-server application (or improve an SSL based one). Now it certainly would be a great idea at this point to spend some time evaluating the different methods of securing an applications communication, I could waste pages explaining about SSL or teaching you how to write your proprietary encryption process, but that's not why were here is it? And it's certainly not why I am here! You're here because you know SSH, you use it everyday to access every other computer your work on, or at least you should! You're here because SSH provides strong encryption, integrity checking and public key authentication of both the client and the server; the thing is you want your client-server application to provide that too, so that's what were going to do today.

This tutorial assumes you have a working knowledge of SSH clients and servers. Most importantly you should understand the concept of SSH port forwarding. In this series of articles we will discuss two ways to secure your client-server application

1. Secure existing applications by channelling the TCPIP data through a secure tunnel to an SSH server installed on the same machine as your application server

You may have already used standard SSH port forwarding to setup a secure tunnel to transport your connection, but maintaining both the SSH client and SSH server in addition to your own client and server is not a realistic proposition.

The quickest way to secure your application with J2SSH is to use this very process, but instead of maintaining an SSH client, you can use J2SSH directly within your client code.

2. Build an SSH server to provide your applications services.

J2SSH comes with a set of daemon classes that enable the creation of a custom SSH server in no time! The SSH server itself is already preconfigured to accept connections so the main framework is already available. Your responsibility firstly will be to develop an authentication provider to plug into your existing applications authentication mechanism, and secondly to provide a channel implementation that replaces your applications socket communication. There are of course a few more details but it could not be much simpler!

Securing existing applications by channelling TCPIP data through a secure tunnel

This method essentially forwards your client-server communication through a secure tunnel just as if you had setup standard port forwarding between your client host and the server. The difference is that J2SSH does not require the installation of an SSH client, instead it allows your client application to connect to the SSH server directly and instruct the server where to forward the data to thus removing the configuration nightmare of using port forwarding.

Setup an SSH server on the same host as your application server

First thing we need to do is to install an SSH server on the same host as your application server. That's up to you I'm afraid, we cannot help you out there. Once you have the server installed create an account on the host for your client application, J2SSH will need to authenticate to the SSH server, so it's a good idea to have an account for the application, and since we don't want to be putting passwords into the code we will also want a private key without a passphrase.

Configure your applications account with the public key and test the connection; you should be able to connect without any prompting. Finally, setup your application server to listen on the local loopback address. This will ensure that clients cannot connect directly.

Replace your existing client socket connection with an SSH connection using J2SSH

Now that we have a server installed, a private key for your application to authenticate with and your application server running, we can now turn to the client code changes. We need to replace your existing connection code with the following process

Create a connection to the SSH server
Authenticate using the client applications public key
Create a tunnel to your application server
Start your own communication using the tunnel's IO streams

Creating the connection is straight forward, you will obviously need to take into account integrating this code into your own client but the connection procedure is as follows

import com.sshtools.j2ssh.SshClient;

public SshClient createConnection() {
// Make a client connection SshClient
ssh = new SshClient();
// Connect to the host
ssh.connect("your-applications-host");
}

If you run the above code you will notice that the user is prompted in the console to verify the host key of the server. Its probably going to be a good idea to override this default behaviour since not everyone will see the console output. There are a couple of methods available, you could use a different HostKeyVerification implementation, for example the DialogHostKeyVerification will prompt the user using a dialog, and there is also IgnoreHostKeyVerification which simply ignores the process. For now we will use this, since were sure that the server is correct, however when you deploy your client to production environments its wise to replace this with either the dialog verification or your own custom implementation.

Lets import the new verification process

import com.sshtools.j2ssh.transport.IgnoreHostKeyVerification;

and change the connect call

ssh.connect("your-applications-host", new IgnoreHostKeyVerification());

Now we can connect to the SSH server, and the next step is to authenticate using our newly generated private key. First we need to open the private key file, this can be done with the following code.

import com.sshtools.j2ssh.transport.authentication.SshPrivateKeyFile; .... // Open up the private key file SshPrivateKeyFile file = SshPrivateKeyFile.parse(new File("filename"));

This provides us with a copy of the file in memory, since we don't have a passphrase on the key we can simply add the key to a new instance of the PublicKeyAuthenticationClient which we are going to use to authenticate with.

import com.sshtools.j2ssh.authentication.PublicKeyAuthenticationClient; ..... PublicKeyAuthenticationClient pk = new PublicKeyAuthenticationClient(); Pk.setUsername("username"); Pk.setKey(file.toPrivateKey(null));

Now that the key is loaded into the authentication instance, we can authenticate.

if(ssh.autenticate(pk)== AuthenticationProtocolState.COMPLETED) {
// Authentication Complete
}

Now that we have successfully authenticated, were ready to create the secure tunnel for your application. To do this we are going to create a local forwarding channel, local forwarding is initiated by the client where by remote forwarding is initiated by the server. This forwarding channel will provide a set of IO streams for your client application to use in place of the Sockets IO streams.

import com.sshtools.j2ssh.forwarding.ForwardingIOChannel; ..... ForwardingIOChannel channel = new ForwardingIOChannel(
ForwardingIOChannel.LOCAL_FORWARDING_CHANNEL,
"127.0.0.1",
10022,
"myhost",
10022);

We now have a channel ready so lets waste no more time and open it

if(ssh.openChannel(channel)) {
// Channel is opened and can be used
channel.getOutputStream("START".getBytes());
....
}

Once the channel is open, you can use its IO streams in place of your Socket, your application should not require any changes.

The advantages and disadvantages of using a secure tunnel

The main advantage of this method is that it's a quick fix for an insecure application. By forwarding the traffic through an embedded SSH tunnel your application's data is now secure. However, your server has no access to the strong authentication that the SSH server provides and you still need to maintain a separate SSH server and authentication details for your client application. Of course these can be minimized by embedding a private key into your client application but that in itself a potential security flaw. What you need is to have your server support SSH connections directly.

In the next part of this series, I will show you how to build your own SSH server using the J2SSH daemon classes. Be prepared though as its not something you're going to be able to plug-in to your existing server, instead we will build an SSH server capable of supporting your applications proprietary protocol.

Convert Between Base 10 and Base 62 in PL/SQL

2008-09-05T08:34:00.000-07:00

There are many reasons why you may find yourself needing to convert a value back and forth between base 10, base 62, base 36, hexadecimal or whatever. I found myself in a similar situation where I had to generate a unique id entries in a data file, yet I only had two characters where I could put this value. Since this file could possibly have up to 3,000 entries in it, I was not going to be able to just use numbers since that only gives me 10 x 10 = 100 possible values ( 0 to 99 ).

However, if I were to translate a sequential decimal ( base 10 ) number to base 62, I now have 62 x 62 = 3,844 possible values ( 0 - zz ). Now obviously for base 62, the value will be case sensitive. If you can't guarantee case sensitivity, then you'll have to use base 36.

Convert from Base 10 to Base 62 in PL/SQL

CREATE OR REPLACE FUNCTION f_to_base62( a_number_to_convert INTEGER ) RETURN VARCHAR2 IS
  
 v_modulo INTEGER;
 v_temp_int INTEGER := a_number_to_convert;
 v_temp_val VARCHAR2(256);
 v_temp_char VARCHAR2(1);
 c_base62_digits CONSTANT VARCHAR2(62) := '0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz';

BEGIN

 IF ( a_number_to_convert = 0 ) THEN
   v_temp_val := '0';
 END IF;
      
 WHILE ( v_temp_int <> 0 ) LOOP
   v_modulo := v_temp_int MOD 62;
   v_temp_char := SUBSTR( c_base62_digits, v_modulo + 1, 1 );
   v_temp_val := v_temp_char || v_temp_val;
   v_temp_int := floor(v_temp_int / 62);
 END LOOP;

 RETURN v_temp_val;

END;

Convert from Base 62 to Base 10 in PL/SQL

CREATE OR REPLACE FUNCTION f_from_base62( a_value_to_convert VARCHAR2 ) RETURN INTEGER IS

 v_iterator INTEGER;
 v_length INTEGER;
 v_temp_char VARCHAR2(1);
 v_temp_int INTEGER;
 v_return_value INTEGER := 0;
 v_multiplier INTEGER := 1;
 v_temp_convert_val VARCHAR2(256) := a_value_to_convert;
 c_base62_digits CONSTANT VARCHAR2(62) := '0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz';

BEGIN

 v_length := length( v_temp_convert_val );
 v_iterator := v_length;
 WHILE ( v_iterator > 0 ) LOOP
   v_temp_char := SUBSTR( v_temp_convert_val, v_iterator, 1 );
   v_temp_int := INSTR( c_base62_digits, v_temp_char ) -  1;
   v_return_value := v_return_value + ( v_temp_int * v_multiplier );
   v_multiplier := v_multiplier * 62;
   v_iterator := v_iterator - 1;
 END LOOP;

 RETURN v_return_value;

END;

Getting Started with J2SSH

2008-09-03T04:02:00.000-07:00

The first thing you will probably want to do is to connect to an SSH server using J2SSH. This is a fairly
straightforward procedure using the SshClient class. This class provides access for connecting, authenticating
and starting a session channel, which enables you to execute commands or start the users shell.

import com.sshtools.j2ssh.SshClient;

First of all prepare your application, in this section we will guide you through the basics so for now just a simple
try/catch inside the static main method.

import java.io.BufferedReader;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.io.OutputStream;



public class SshExample() {
// A buffered reader so we can request information from the user
private static BufferedReader reader =
     new BufferedReader(new InputStreamReader(System.in));

public static void main(String args[]) {
 try {
   // Further code will be added here

} catch(Exception e) {
   e.printStackTrace();
 }
}
}

The next few sections will guide you through making the initial connection, authenticating the user and executing
a command or starting the users shell for a simple console based SSH application.

Making the initial connection

To create an SshClient instance import the class into your implementation class file and use the following code
to connect to an SSH server on the standard port 22.

SshClient ssh
= new SshClient();


System.out.print("Host to connect: ");

String hostname = reader.readLine();

ssh.connect(hostname);

When the client connects to the server, the server supplies its public key for the client to verify. You will see
that calling the connect method prompts the user within the console to verify the key:

The host firestar is currently unknown to the system The host key fingerprint is: 1028: 69 54 9c 49 e5 92 59 40 5 66 c5 2e 9d 86 af ed Do you want to allow this host key? [Yes|No|Always]:

In the default implementation of the connect method, J2SSH reads the $HOME/.ssh/known_hosts file to determines to which hosts connections
may be allowed. This is provided by the class ConsoleKnownHostsKeyVerification and the
default behavior can be emulated by the following code:


import com.sshtools.j2ssh.transport.ConsoleKnownHostsKeyVerification;


ssh.connect("firestar", new ConsoleKnownHostsKeyVerification());

When the connect method returns, the protocol has been negotiated and key exchange has taken place, leaving the connection ready for authenticating the user.

Authenticating the user

Once the connection has been completed the user is required to provide a set of credentials for authentication.
All client side authentication methods are implemented using the abstract class:

import com.sshtools.j2ssh.authentication.SshAuthenticationClient.

To perform authentication, the SshClient class provides the following method:

public int authenticate(SshAuthenticationClient auth);

There are currently five authentication methods implemented by J2SSH, 'password', 'publickey', 'keyboard-interactive' and 'hostbased'. With an extra agent authentication method that performs public key authentication using the J2SSH key agent.

Password Authentication

Password authentication is ideal for first time users as it requires no additional configuration within the SSH
client or server. The user simply supplies his username and password to the client which is then transmitted over
the encrypted connection to the server. The server then checks that the given password is acceptable to the native
password-authentication mechanism of the host operating system and returns the result to the client.

J2SSH implements the 'password' authentication method with the following class:

import com.sshtools.j2ssh.authentication.PasswordAuthenticationClient

Using the password authentication method is straight forward; create an instance of the PasswordAuthentication
class, set the username and password and pass to the SshClient to complete the authentication.

 /**
* Create a PasswordAuthenticationClient instance, set the properties
* and pass to the SessionClient to authenticate
*/
PasswordAuthenticationClient pwd = new PasswordAuthenticationClient();

System.out.print("Username: ");
String username = reader.readLine();
auth.setUsername(username);

System.out.print("Password: ");
String password = reader.readLine();
auth.setPassword(password);

int result = ssh.authenticate(pwd);

The Authentication Result

When the authentication method completes it returns the result of the authentication. This integer value can be any
of the following three values defined in the class:

import com.sshtools.j2ssh.authentication.AuthenticationProtocolState;
..
..
if(result==AuthenticationProtocolState.FAILED)
  System.out.println("The authentication failed");

if(result==AuthenticationProtocolState.PARTIAL)
  System.out.println("The authentication succeeded but another"
                  + "authentication is required");

if(result==AuthenticationProtocolState.COMPLETE)
  System.out.println("The authentication is complete");

Retrieving the available authentication Methods

It is possible at any time after the connection has been established and before authentication has been completed to request a list of authentication methods that can be used. The getAvailableAuthMethods method returns a list of authentication method names.

public List getAvailableAuthMethods(String username);

It should be noted that the SSH specification allows the server to return authentication methods that are not valid
for the user.

Prompting the User for Authentication Details?

Each SshAuthenticationClient implementation can optionally be set a prompt interface which allows the user to
be prompted for the information once the authenticate method has been invoked.

public interface SshAuthenticationPrompt {
public boolean showPrompt(SshAuthenticationClient instance)
                           throws AuthenticationProtocolException;
}
}

The showPrompt method is called if the authentication instance is not ready to authenticate. The methods is called and the developer should verify the instance of the SshAuthenticationClient to make sure that it is compatible with the prompt (for example you cannot perform public key authentication
with a password prompt!). The user should then be duly prompted for the information and the instance set with the user's information. Once
complete, the prompt returns true to indicate that the user successfully entered correct information.

There are several prompts provided in the J2SSH common packages that provide useful Swing based dialogs to
prompt the user.


import com.sshtools.common.authentication.PasswordAuthenticationDialog;

/***
* Create a PasswordAuthenticationDialog instance and call the
* showAuthenticationMethod so the user can graphically
* enter their username and password
*/


PasswordAuthenticationClient pwd =
         new PasswordAuthenticationClient();

PasswordAuthenticationDialog dialog =
         new PasswordAuthenticationDialog(parent);

pwd.setAuthenticationPrompt(dialog);

int result;
result = ssh.authenticate(pwd);

Using a session channel to execute a command or start the users shell

Once the user is authenticated you will probably want to do something such as execute a command or start the users shell. The SSH protocol provides multiplexed channels over a single connection and the session channel is one of the channels defined by the SSH protocol. The session channel allows the client to execute a single command on the remote host and to communicate with the process by sending and receiving data. The J2SSH SessionChannelClient implements this channel and we will use this to execute a basic "ls" command.

The session channel provides an inputstream and outputstream for reading/writing, but before we can do this we need to setup the channel for our command. First we open the channel itself by calling the SshClient method:


SessionChannelClient session = ssh.openSessionChannel();

Now that we have a session instance we need to configure it for our command, there are several options that can be set before we invoke one of the methods that will start the session.

Setting Environment Variables
The SSH protocol provides a method to set an environment variable for the processes environment, the protocol also leaves the actual implementation of this down the server implementation, and in our experience most servers do not allow for this for security reasons. However the method is available since its defined in the protocol specification, so your free to try to use it, but beware the variable may not be set!

public boolean setEnvironmentVariable(String name, String value);

Requesting a Pseudo Terminal
A pseudo terminal is a device that imitates a terminal. Rather than being connected to an actual terminal, a pseudo-terminal (or pty) is connected to a process. If the command you are executing is expecting a terminal (such as a shell command) you can request that a pseudo terminal be attached to the process by calling the requestPseudoTerminal method.

public boolean requestPseudoTerminal(String term, int cols, int rows, int width, int height, String modes);

Invoking a command
After the above operations have been performed you can then request that the session either start the user's shell, execute a specific command or start an SSH subsystem (such as SFTP). You should not invoke a subsystem unless you are able to read/write the subsystem protocol, there are many additional utilities within J2SSH that provide for the available subsystems.

To start the users default shell use:

public boolean startShell();

Or to execute a specific command use:

public boolean executeCommand(String command);

An important note to remember is that this does not execute a shell command. You cannot for instance issue the command executeCommand("dir")" on the Windows Operating system as this is a shell command, instead use "cmd.exe /C dir". This method executes a binary executable and so should be used to execute any program other than the users shell.

Handling Session Data
Once the session has been configured and a command or shell has been started, you can begin to
transfer data to and from the remote computer using the sessions IO streams. These streams provide you with a
standardized interface for reading and writing the data.

The Session Channel's OutputStream
The format of writing data varies according to how you configured the session, for example if you executed the
users shell then the data should be written as if the user had entered the commands interactively.


/** Writing to the session OutputStream */
OutputStream out = session.getOutputStream();
String cmd = "ls\n";
out.write(cmd.getBytes());

The Session Channel's InputStream

/**
* Reading from the session InputStream
*/
InputStream in = session.getInputStream();
byte buffer[] = new byte[255];
int read;
while((read = in.read(buffer)) > 0) {
String out = new String(buffer, 0, read);
System.out.println(out);
}

Reading from stderr
The session also provides the stderr data provided by the remote session. Again an InputStream is provided.

public InputStream session.getStderrInputStream();

Closing the Session

The session can be closed using the following method:

public void close();

Disconnecting

The connection can be terminated by either side. To terminate the connection call the SshClient method:

public void disconnect();

A word on executing multiple commands

So we can now execute a single command on the remote server, but what's that I hear you say? I want to execute more than one command? Well if you cast your mind back I told you that the SSH protocol provides multiplexed channels over a single connection, so executing another command is as simple as executing the first, just create a new instance of the SessionChannelClient for every command you want to execute. You can execute them simultaneously or one after another, but always create a new session (since the session is closed when the command finishes and the protocol does not allow for re-using of a session to execute another command).

There is a drawback to this in that the process environment is not passed on from one session to another, so you cannot for example execute a command to change directory and then another to execute a script in that directory, since the change directory is lost when the session closes and the new command starts back in the default working directory. Of course you could always put the cd command into the script? Or use the shell to execute both commands. This subject is certainly a bit more advanced so I will leave it for another day and pencil in a new article to discuss all the alternatives for executing multiple commands.

This concludes our getting started tutorial, you should now have a basic working knowledge of how to connect, authenticate and execute commands using J2SSH.

Automate Data Transfer and Web Actions Using cURL

2008-08-31T09:02:00.000-07:00

If you have ever had the need to automate some action such as transferring files using a secure protocol such as SFTP, or if you have ever thought that it would be nice to automate some sort of web action, you need to know about cURL. According to their website:

curl is a command line tool for transferring files with URL syntax, supporting FTP, FTPS, HTTP, HTTPS, SCP, SFTP, TFTP, TELNET, DICT, LDAP, LDAPS and FILE. curl supports SSL certificates, HTTP POST, HTTP PUT, FTP uploading, HTTP form based upload, proxies, cookies, user+password authentication (Basic, Digest, NTLM, Negotiate, kerberos...), file transfer resume, proxy tunneling and a busload of other useful tricks.

I have actually used cURL in a few projects now and I am very impressed with it's rich set of features and reliability to automate tasks. I won't go over every possible use for curl, but I'll show one example of how I have used cURL in a past project.

I recently setup a text messaging alert using Google's SMS system and Microsoft Outlook. Based on certain types of messages, Outlook will kick off a batch file, that contains one command:

curl -d mobile_user_id="555-555-1234" -d carrier="ATT" -d text="TEXT MESSAGE DATA"
-d gl="US" -d hl="en" -d client="navclient-ffsms" -d from="" -d source="" -d c="1"
http://www.google.com/sendtophone

In the command above, all of the -d parameters are post parameters that will be sent to the Google SMS site and they are fairly self explanatory. After the parameters, we have the URL http://www.google.com/sendtophone which is where the Google SMS form is located.

If you are looking for other ways to use cURL, check out their manual.

PL/SQL MERGE Query ( UPSERT )

2008-08-30T16:53:00.000-07:00

One of my recent findings about PL/SQL is the MERGE query, also known as an UPSERT. This is exactly what it sounds like, an INSERT and UPDATE all in one block of code. This can be really useful and can really reduce the amount of code you have to write to perform certain operations. On top of reducing the amount of code, it can really make your code more readable and easier to maintain.

Basic MERGE Syntax

MERGE INTO destination_table
USING (SELECT key_column, value_column FROM source_table)
ON (destination_table.key_column = source_table.key_column)
WHEN MATCHED THEN
 UPDATE SET destination_table.value_column = source_table.value_column
WHEN NOT MATCHED THEN
 INSERT (destination_table.key_column, destination_table.value_column)
 VALUES (source_table.key_column, destination_table.value_column);

Looking at the basic syntax above, you get an idea of how the MERGE query works. You can make this as basic or as complicated as you would like. Of course, you can use JOINS, VIEWS, etc. in either the source or destination queries. You can leave off either of the WHEN MATCHED THEN or WHEN NOT MATCHED THEN blocks if you don't want to perform operations in those certain situations.

Let's take a look at an example scenario where the MERGE query could be used. In our example, we have two tables. In one table, we maintain a list of widgets and in another table we keep the properties for those widgets. The tables with some sample data is listed below.

ID	WIDGET
1	Super Widget
2	Awesome Widget

WIDGET_ID	PROP_NAME	PROP_VALUE
1	COLOR	GREEN
1	SIZE	LARGE
2	SIZE	MEDIUM

In our WIDGET table, we have two widgets. The Awesome Widget and the Super Widget. In our WIDGET_PROPERTY table, we have two properties for the Awesome Widget, size and color. We only have one property for the Super Widget, which is size. Management has decided that all widgets will now come in the color blue. So we need to have a color property for all widgets and the value must be blue.

This could get pretty complicated since some of the widgets already have a color property and we can't add another color property to a given widget. So a simple INSERT won't work unless we delete all color properties first, which isn't very efficient, especially if there is a lot of data.

We could use an UPDATE query for the ones that already exist, but not all widgets have a color property, so this will still not solve that problem. With a MERGE query, we can solve this problem with a single statment in PL/SQL. Here is the PL/SQL code we would use to handle this situation.

MERGE INTO WIDGET_PROPERTY DESTINATION
USING ( SELECT WIDGET.ID, 'COLOR' AS PROP_NAME FROM WIDGET ) SOURCE
ON ( DESTINATION.WIDGET_ID = SOURCE.ID AND
    DESTINATION.PROP_NAME = SOURCE.PROP_NAME )
WHEN MATCHED THEN
  UPDATE SET DESTINATION.PROP_VALUE = 'BLUE'
WHEN NOT MATCHED THEN
  INSERT ( DESTINATION.WIDGET_ID, DESTINATION.PROP_NAME, DESTINATION.PROP_VALUE )
  VALUES( SOURCE.ID, SOURCE.PROP_NAME, 'BLUE' );

Associative Arrays: Hashtables for PL/SQL

2008-08-29T16:42:00.000-07:00

I have been developing software in Java most of my career, but recently my job has required me to take on more projects using Oracle's PL/SQL. Being a Java programmer, I am very well versed in the various collections that are available from the Java API. One collection I find myself using a lot is the Hashtable and recently during a PL/SQL project, I had an algorithm that would be served nicely by such a data structure.

That's when I found out about Associative Arrays in PL/SQL. An associative array is a key-value pair that we can use in very much the same fashion we would use the Hashtable type in Java. In this post I'll show you how to declare a PL/SQL associative array, assign values to the array, then how to access those values. Then I'll go over how to iterate over the PL/SQL associative array.

Declaring, Assigning, and Accessing Associative Arrays

DECLARE

  -- Declare our array type to be keyed on 5 byte varchar and hold values of 256 byte varchar.
  TYPE zip_city_type IS TABLE OF VARCHAR2(256) INDEX BY VARCHAR2(5);
  -- Declare our actual associative array using our new data type.
  zip_city_array zip_city_type;

BEGIN

  -- Let's add some values to our array.
  zip_city_array('80201') := 'DENVER, CO';  -- Adds entry for zip code 80201
  zip_city_array('48201') := 'DETROIT, MI';

  -- Print out the values from our associative array.
  dbms_output.put_line( 'Zip code 80201 is in ' || zip_city_array('80201') );
  dbms_output.put_line( 'Zip code 48201 is in ' || zip_city_array('48201') );

END;

Iterating Through an Associative Array

You will come across situations where you need to iterate through all of the elements in your associative array and you want to do so efficiently. Let's take a look at how we would accomplish iterating through the array we declared above.

DECLARE

  -- Declare our array type to be keyed on 5 byte varchar and hold values of 256 byte varchar.
  TYPE zip_city_type IS TABLE OF VARCHAR2(256) INDEX BY VARCHAR2(5);
  -- Declare our actual associative array using our new data type.
  zip_city_array zip_city_type;
  zip VARCHAR2(5);

BEGIN

  -- Let's add some values to our array.
  zip_city_array('80201') := 'DENVER, CO';  -- Adds entry for zip code 80201
  zip_city_array('48201') := 'DETROIT, MI';

  -- Assign the first key in our array to zip.
  zip := zip_city_array.FIRST;  
 
  -- Loop through the array while our key is null.
  WHILE zip IS NOT NULL LOOP
     dbms_output.put_line( zip || ' = ' || zip_city_array(zip) );
     zip := zip_city_array.NEXT( zip );
  END LOOP;  

END;

Java Source Reflection with JaxMe JavaSource

2008-08-25T16:46:00.000-07:00

Alright, it may not happen very often that you need to be able to use reflection on your Java source code, but its nice to know there is a framework that can help you out. Apache's JaxMe JavaSource framework is a framework that can be used for generating Java source files.

I personally found myself working on a project that had to be capable of integrating our source code with another vendor's source code, both of which changed frequently. I was able to use the JaxMe JavaSource framework to verify that all necessary methods had been implemented on our subclasses, and if they hadn't, I was able to stub them out quickly and easily so that it didn't break the build everytime a base class was modified.

For more information on how to use the JaxMe JavaSource framework and in particular the source code reflection portion of the framework, go here.

Introducing BeanShell: A Java Source Interpreter

2008-08-24T05:23:00.000-07:00

If you haven't ever checked out BeanShell before, I would highly suggest you mosey on over to www.beanshell.org and take a look. They describe BeanShell as a small, free, embeddable Java source interpreter with object scripting language features, written in Java.

Essentially, it is a way to run Java source code as an interpreted script. Along with supporting the full Java syntax, it adds a few other nice scripting features such as loosely typed variables. Some of the benefits of such an application include rapid prototyping, rules engines, configuration, testing, embedded systems, and even Java education.

Also, if your application is using Spring, you can even inject BeanShell scripts into your Spring beans. They provide a quick and easy tutorial to get you up and running with BeanShell here.

Injecting BeanShell Scripts into a Spring Bean

2008-08-23T18:54:00.000-07:00

One really neat feature about Spring that you don't hear very much about is the ability to inject scripting into a Spring bean. This is very powerful functionality that allows you to add dynamic logic into your application and even has the ability to reload the script at a set interval enabling you to make code changes on the fly without rebuilding. There are currently three scripting languages that you can use with this feature and they are Groove, Ruby, and BeanShell. This post will focus on injecting a BeanShell script into a Spring bean.

We are going to take a look at a situation in which we are going to implement various rules for calculating invoice amounts for ACME Consulting Company's clients. Some of ACME's clients pay a set monthly fee that entitles them to a given number of hours of service, then they pay an hourly rate for any hours over the set amount. They also have some clients that only pay an hourly fee for service and don't have a monthly contract setup.

For this situation, we are going to setup two invoice calculation rules, one for calculating invoice amounts for our monthly contracts, and one that only calculates invoices for our hourly contracts.

Before we take a look at the code, you'll need to add the BeanShell library to your class path which can be downloaded from www.beanshell.org.

First, we'll create a Java class called InvoiceData that will hold all of the information our rules will need to calculate the invoice amounts:


package com.billing.invoice;

import java.math.BigDecimal;

public class InvoiceData {

   private int hoursWorked;
   private int hoursEntitled;
   private BigDecimal monthlyRate;
   private BigDecimal hourlyRate;

   public void setHoursWorked( int hours ) {
        hoursWorked = hours;
   }

   public void setHoursEntitled( int hours ) {
        hoursEntitled = hours;
   }

   public void setMonthlyRate( BigDecimal rate ) {
        monthlyRate = rate;
   }

   public void setHourlyRate( BigDecimal rate ) {
        hourlyRate = rate;
   }

   public int getHoursWorked() {
        return hoursWorked;
   }

   public int getHoursEntitled() {
        return hoursEntitled;
   }

   public BigDecimal getMonthlyRate() {
        return monthlyRate;
   }

   public BigDecimal getHourlyRate() {
        return hourlyRate;
   }

}

This is your basic plain old java object (POJO) that will hold the properties we will need to calculate the total invoice amount.

Next, we'll create a Java interface called InvoiceCalculationRule:


package com.billing.invoice.rules;

import java.math.BigDecimal;
import com.billing.invoice.InvoiceData;

public interface InvoiceCalculationRule {

   public BigDecimal calculate( InvoiceData invoiceData );

}

This interface contains only one method named calculate, that accepts an invoiceData object. This interface will be used to perform our invoice calculations. The actual implementations are coming up next and they are our BeanShell scripts.

Now we are ready to build our rule classes. These will be BeanShell scripts, but the syntax is exactly the same as Java which is another reason BeanShell is a great option; no need to learn a different language to implement scripting.

Our first rule, MonthlyContractsInvoiceCalculationRule will calculate invoices for our monthly contracts:


import java.math.BigDecimal;
import com.billing.invoice.InvoiceData;

public BigDecimal calculate( InvoiceData invoiceData ) {

   int chargeableHours = invoiceData.getHoursWorked() - invoiceData.getHoursEntitled();
   BigDecimal chargeableHoursAmount = new BigDecimal( 0 );
   BigDecimal invoiceAmount = new BigDecimal( 0 );

   if ( chargeableHours > 0 ) {
        chargeableHoursAmount = invoiceData.getHourlyRate().multiply( new BigDecimal( chargeableHours ) );
   }

   invoiceAmount = chargeableHoursAmount.add( invoiceData.getMonthlyRate() );

   return invoiceAmount;

}

Our second rule, HourlyContractsInvoiceCalculationRule will calculate invoices for our hourly contracts:


import java.math.BigDecimal;
import com.billing.invoice.InvoiceData;

public BigDecimal calculate( InvoiceData invoiceData ) {

   BigDecimal invoiceAmount = invoiceData.getHourlyRate().multiply(
        new BigDecimal( invoiceData.getHoursWorked() ) );

   return invoiceAmount;

}

Now that we have all of our classes setup, we need to work on our Spring configuration file so that we can actually retrieve our rules when we need them. Here's an example of what our configuration file would look like:


<beans xmlns="http://www.springframework.org/schema/beans"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xmlns:lang="http://www.springframework.org/schema/lang"
    xsi:schemaLocation="http://www.springframework.org/schema/beans
         http://www.springframework.org/schema/beans/spring-beans-2.0.xsd    
         http://www.springframework.org/schema/lang
         http://www.springframework.org/schema/lang/spring-lang-2.0.xsd">

    <lang:bsh id="hourlyContractsInvoiceCalculationRule"
         script-source="classpath:com/billing/invoice/rules/HourlyContractsInvoiceCalculationRule.bsh"
         script-interfaces="com.billing.invoice.rules.InvoiceCalculationRule"
         refresh-check-delay="60000" />

    <lang:bsh id="monthlyContractsInvoiceCalculationRule"
         script-source="classpath:com/billing/invoice/rules/MonthlyContractsInvoiceCalculationRule.bsh"
         script-interfaces="com.billing.invoice.rules.InvoiceCalculationRule"
         refresh-check-delay="60000" />

</beans>

We have everything in place and we are ready to tie it all together. For this example, I am just writing up a quick test, so it isn't a realistic class, but it will give you an idea of how to use our new rules.


package com.billing.invoice;

import com.billing.invoice.rules.InvoiceCalculationRule;

import java.math.BigDecimal;
import java.math.RoundingMode;

import org.springframework.context.ApplicationContext;
import org.springframework.context.support.ClassPathXmlApplicationContext;

public class RuleTestMain {

     public static void main( String [] args ) {

          ApplicationContext context = new ClassPathXmlApplicationContext( "spring.xml" );
          InvoiceData invoiceOne = new InvoiceData();
          InvoiceData invoiceTwo = new InvoiceData();

          invoiceOne.setHoursWorked( 40 );
          invoiceOne.setHoursEntitled( 30 );
          invoiceOne.setMonthlyRate( new BigDecimal( 1500 ) );
          invoiceOne.setHourlyRate( new BigDecimal( 44.25 ) );

          InvoiceCalculationRule monthlyRule =
               ( InvoiceCalculationRule ) context.getBean( "monthlyContractsInvoiceCalculationRule" );

          System.out.println( "Monthly Invoice Amount: $" + monthlyRule.calculate( invoiceOne ).setScale( 2, RoundingMode.HALF_UP ) );

          invoiceTwo.setHoursWorked( 40 );
          invoiceTwo.setHourlyRate( new BigDecimal( 52.25 ) );

          InvoiceCalculationRule hourlyRule =
               ( InvoiceCalculationRule ) context.getBean( "hourlyContractsInvoiceCalculationRule" );

          System.out.println( "Hourly Invoice Amount: $" + hourlyRule.calculate( invoiceTwo ).setScale( 2, RoundingMode.HALF_UP ) );

     }

}

When we run this code, we will be calculating the first invoice using the monthly rule, then the second invoice will be calculated using the hourly rule. The great thing is, that if we decide we need to change one of these rules, we can actually update the scripts and it will be automatically reloaded every minute and will become active without having to rebuild the application.

The often unappreciated MINUS SQL query

2008-08-23T17:29:00.000-07:00

There are times when you need to verify that a given record is associated with all of a given type of records. In this post, I'll be going over the MINUS statement and show you a practical use for this often under used functionality of SQL.

For this example we'll be looking at customer data for an online application. ACME paper in has unleashed a new paper sales website. They allow the employees of the company to which they sale paper to have access to various sections of their website. In order to have more granular control, they control access the sections at both the user level and the company level. In order for a user to have access to a section, they must both have access at the company level and at the user level. Let's take a look at our tables to get a better idea of what we are dealing with.

ID	COMPANY_NAME
1	Awesome Corp.

ID	USERNAME
1	jdoe
2	bsmith

ID	SECTION_NAME
1	Main
2	Special

COMP_ID	USER_ID
1	1
1	2

COMP_ID	SECT_ID
1	1
1	2

USER_ID	SECT_ID
1	1
1	2
2	1

The first table (COMPANY) shows the companies that ACME does business with. The second table (USER) shows the users that are using the new online sales site. The third table (SECTION) shows the various sections that are available on the online sales site. The fourth table (COMPANYUSERJOIN) links the users to the various companies. The fifth table (COMPANYSECTIONJOIN) gives company access to the various sections. The sixth table (USERSECTIONJOIN) gives user access to the various sections.

Management has requested a report of all of the users that don't have access to all of the sections that their company has access to. In comes MINUS to save the day. Minus will return only the rows that are not in both queries. Just like UNION, the data types in both SQL queries must be the same. Let's take a look at the query that will return the data we are looking for.

SELECT SECTION.ID, USER.USERNAME FROM USER
JOIN COMPANYUSERJOIN ON COMPANYUSERJOIN.USER_ID = USER.ID
JOIN COMPANYSECTIONJOIN ON COMPANYSECTIONJOIN.COMP_ID = COMPANYUSERJOIN.COMP_ID
JOIN SECTION ON SECTION.ID = COMPANYSECTIONJOIN.SECT_ID
MINUS
SELECT SECTION.ID, USER.USERNAME FROM USER
JOIN USERSECTIONJOIN ON USERSECTIONJOIN.USER_ID = USER.ID

Looking at the SQL query above, we can see in the first section that we are selecting the section id and the username by joining the section table through what the corporation will have access to. This will return all of the usernames for the corporations, along with all of the section ids the corporation has access to. Then we have minus before the second part of the query. In the second part of the SQL query, we are returning the section ids and usernames based on what section each user actually has access to.

The minus will compare these two SQL queries and remove duplicate records. This will return two types of records. The first type of record returned will be the sections that a company has access to, but that a given user has not been given access to. The second type of record that will be returned will be sections that a user has access to, but that a given corporation does not. For our exercise, we will assume that a given user can't be given access to a section without the company also having access.

Using the example data above, this query will return a section id of 2 and username of bsmith. This means that bsmith is part of a company that has access to section 2, but bsmith does not have access to this section. We then write up our report for management, cut out of work early, and go grab a cold beer or two with some friends. Thanks to the SQL MINUS query.

Delete multiple lines in a text file using regular expressions and sed

2008-08-23T08:45:00.000-07:00

I recently found myself in quite a predicament. I had a very large flat file in a specific format and I needed to delete certain record types. Now, obviously this is very easy with regular expressions if the records are only on one line. However, that was not the case. The record types I needed to remove took up multiple lines. Below is an example of the type of record I needed to delete (however since the file spec is proprietary I have changed the actual values):

START PAYMENT_RECORD
123456789          80000   JOHN    DOE    BANK OF ANYWHERE USA    123 ANY STREET  ANYTOWN USA    
END PAYMENT_RECORD

However with sed, you can still do a multiple line search and delete from a file with the following command:

sed '/pattern/{N;N;N;d}' filename

So using the sed command in my particular record example, I accomplished this task with

sed '/^START PAYMENT_RECORD/{N;N;N;d}' payment_data_file.txt

This command tells sed to find the text START PAYMENT_RECORD at the beginning of the line, then delete that line and the next two lines. The N portion is the number of lines to be removed, and the d portion tells sed to delete the data matching the pattern.

Check if a value is in a PL/SQL collection using MEMBER OF

2008-08-21T10:14:00.001-07:00

You may find yourself working with a collection in PL/SQL and think to yourself, "Wow, it sure would be great if I could do a quick search of this collection to see if it contains a specific value." Well, never fear, cause I'm going to show you how to do just that using MEMBER OF.

ID	NAME
1	Black
2	Brown
3	Blue
4	Green
5	Orange
6	Red
7	Purple
8	Yellow

Let's say that you have the names of the eight basic colors (according to Crayola) in a lookup table called LKBASICCOLORS that has only two columns, ID and NAME.

In our situation, we want to verify that a given color name is one of our eight basic colors. There are numerous ways to do this in PL/SQL, including hard coding these color names into our code. While in this situation that wouldn't be such a bad idea, but in the real world, values tend to change often and nobody wants to be the poor schmuck that gets to go back through all of the PL/SQL looking for hard coded values. By using member of, you can actually look up these values on the fly, load them into a collection, and verify that our given color is a basic color.

DECLARE

 v_color_white varchar2(10) := 'White;
 v_color_purple varchar2(10) := 'Purple';

 TYPE lkbasiccolor_names IS TABLE OF VARCHAR2(20);
 v_color_names lkbasiccolor_names;

BEGIN

 SELECT NAME BULK COLLECT INTO v_color_names FROM LKBASICCOLORS;

 IF v_color_white MEMBER OF v_color_names THEN
 dbms_output.put_line('White is a basic color');
 ELSE
 dbms_output.put_line('White is not a basic color');
 END;

 IF v_color_purple MEMBER OF v_color_names THEN
 dbms_output.put_line('Purple is a basic color');
 ELSE
 dbms_output.put_line('Purple is not a basic color');
 END;

END;

Now let's take a look at what's going on in the sample above. In the DECLARE, we declare two color name variables. Then we define our collection type as a table of varchar2 values. We move on and declare our actual collection as v_color_names as the collection type of lkbasiccolor_names.

In the first line of the BEGIN section, we load all of our color names into our v_color_names collection. The first test we try is to see if the color white is a member of our v_color_names collection. Since it isn't, it will fall into the else and print out that White is not a basic color. In the next block, since Purple is in our basic color list, it should print Purple is a basic color.