Sunday, January 31, 2016

Better String Aggregation since Oracle 11gR2

In my previous blog "String aggregate in Oracle", I mentioned the restriction of that function is no ordering. Actually since Oracle 11g Release 2, Oracle introduced a new function LISTAGG which provides the same functionality, and also with ordering. You can find details here.

Anyway, I'll show some quick examples in this blog too. To achieve the same result, the query below can be used:
    select employee_id,
           LISTAGG(job_id, '|') WITHIN GROUP (ORDER BY job_id) job_list
    from job_history
    group by employee_id;

And below is the result.
    EMPLOYEE_ID JOB_LIST                     
    ----------- --------------------
            101 AC_ACCOUNT|AC_MGR   
            102 IT_PROG             
            114 ST_CLERK            
            122 ST_CLERK            
            176 SA_MAN|SA_REP       
            200 AC_ACCOUNT|AD_ASST
            201 MK_REP             

And you can see every item in job_list is order.

This new function can be used as analytical function as well.

Saturday, January 30, 2016

Spring Framework: Confusions of RowCallbackHandler

I have had some time not writing my blog. Mainly because my previous employer doesn't allow me to access blogspot during the work hour - also I kind of ran out of topics. I would like to write something that is advanced, not only just tutorials. You can find some many entry-level materials repeating each other, but many times, you'll struggle too long when facing a deeper issue. At least I struggled so many times.

Anyway, I tried to restart blogging. And today's topic is about RowCallbackHandler interface in Spring framework.

You can find many tutorials/samples that suggesting you write program as below:

public class RowCallbackTutorial {
    private DataSource dataSource;
    public void query(String sql) {
        JdbcTemplate jdbcTemplate = new JdbcTemplate(dataSource);
        jdbcTemplate.query(sql, new RowCallbackHandler(){
            public void processRow(ResultSet rs) throws SQLException {
                System.out.println("Inside RowCallbackHandler");
while ( rs.next() ) { System.out.println("Got value:" + rs.getObject(1)); } } }); } }
This actually is not right. Mentioned in the Java doc of this interface, method processRow "should not call next() on the ResultSet" - I guess they mean "should not call the next() on the first row of the ResultSet", since the ResultSet has already opened and pointing to first row when this method is called.

Here is a test using Spring embedded database support.

First we prepare a sql script to create table and insert a few rows, "db/init.sql". I have:
    CREATE TABLE customer (
        id         INTEGER PRIMARY KEY,
        name       VARCHAR(30)
    );

    insert into customer values(1, 'cust1');
    insert into customer values(2, 'cust2');
Then we test with the data.

    public static void main(String[] args) {
        EmbeddedDatabase db = new EmbeddedDatabaseBuilder()
        .setType(EmbeddedDatabaseType.DERBY)
        .addScript("db/init.sql")
        .build();
        String sql = "select * from customer";
        RowCallbackTutorial sample = new RowCallbackTutorial();
        sample.setDataSource(db);
        sample.query(sql);
    }
We're expecting the program to output:
    Got value:1
    Got value:2
In fact, the previous program will only show second row: Got value:2. Where is the first row?

The ResultSet has already opened, and the cursor is pointing to first row. The first "next()" will move the cursor to next row. So the correct operation should be:
                do {
                    System.out.println("Got value:" + rs.getObject(1));
                }  while ( rs.next() );
And run the program again, we now get the correct result:
    Got value:1
    Got value:2
Now here is the last question. What if the ResultSet is empty? Will "do {...} while ()" encounter any error?

The answer is, no problem. In such case, the RowCallbackHandler will not be invoked at all. Change the query to "select * from customer where 1=2" for a new test, the output "Inside RowCallbackHandler" will not appear.

Friday, March 01, 2013

SQLPlus connection without tnsnames.ora

You can connect using sqlplus with user/pwd@tnsname, or with tnsname string directly as:



sqlplus  user/pwd@'(DESCRIPTION=(ADDRESS_LIST=(ADDRESS=(PROTOCOL=TCP)(HOST=<host>)(PORT=1521)))(CONNECT_DATA=(SID=<sid>))'


This is cumbersome, and has a lot of problem on unix, since brackets "(" and ")" need backslashes.   Since Oracle 10g, there's a better way:  
  
sqlplus user/pwd@//host:1521/sid



I feel it's very useful, so noted down for future use.

Wednesday, October 24, 2012

Another program to find hot threads in Java VM

Many people know the GUI tools, such as jconsole and JTop, and maybe jvisualvm in latest JDK releases, to monitor and analyze JVM performance.

The tools jconsole together with JTop can be used to indentify threads that take most of the CPU usage. However, they work under GUI environment. In case you work in UNIX, without XServer, plus you don't have the JMX agent started, you may not have such luxury. It may be rare, but did happen in my case: HP UX, no XServer, cannot be attached to another XServer due to production firewall, the JMX agent is not started. Anyway, it's very frustrating.

Here's a tool can benefit the above case: http://weblogs.java.net/blog/brucechapman/archive/2008/03/hot_threads.html. It takes a process id as input, and prints top threads that take most of the CPU usage. In HP UX environemnt, I have to include tools.jar in the classpath:

    ${JAVA_HOME}/bin/java -Xbootclasspath/a:${JAVA_HOME}/lib/tools.jar -jar HotThread.jar 

Without tools.jar, it gives error that "java.lang.NoClassDefFoundError: com/sun/tools/attach/VirtualMachine".
Tried this in development environment, but didn't have chance to try it in production, due to lengthy procedures. Sigh...

Thursday, October 11, 2012

CVS check out on UNIX with ssh

This article describes scenario that both your CVS server and client are on unix boxes, and you want to check out/check in using ssh.

By doing this, you can avoid typing password each time. Other than CVS settings, it mainly discusses how to set ssh login from one unix server to anohter. Here are the steps.

1. On your client unix, you first need to have environment variables below set:
    export CVSROOT=:ext:<your_cvs_account>@<cvs_server_name>:<cvs_root_dir>
    export CVS_RSH=ssh
    export CVS_SERVER=/usr/local/bin/cvs (or where your cvs is installed on server side)
You may put them in .profile file under your home.

2. Generate key pair on client unix using dsa
    cd ~/.ssh ssh-keygen –t dsa 
Just hit “Return” key when prompted with “Enter passphrase”. You’ll then find two new files created: id_dsa and id_dsa.pub.
    cat id_dsa.pub
Copy the content (you'll paste it on server side in next step).

3. Copy the public key to cvs server.
Login to cvs server using your account:
    cd .ssh
    vi authorized_keys
Add a new line at the end, and paste the content (copied from previous step). Save and quit.
Note: it may be authorized_keys2 depends on the ssh version.

4. Do a test on client unix.
    ssh <cvs_server>
It should let you in without asking for password.

5. Now you can use cvs as usual.
    cvs co <cvs_path>

Struts 2: Populate data for page even after validation failed

I recently added a validation interceptor in one of our Struts 2 applications, but got some errors when validation failed. It took me some time to figure out the error since all exceptions have been redirected to a global exception page, and no logging was written.

Anyway, the error is:
The requested list key 'sortTypeList' could not be resolved as a collection/array/map/enumeration/iterator type.
After some investigation, I figured out the reason being that there is a piece of code in the action method to prepare this list to be dispalyed on the jsp page as a drop down list:

    sortTypeList=new ArrayList();
    sortTypeList.add(new KeyValueVO("id", getText("label.id")));
    sortTypeList.add(new KeyValueVO("name", getText("label.name")));

When the validation fails, this piece of code won't run, leaving the list not being populated. Where KeyValueVO is a class contains two Strings: key and value.

To avoid this, Struts 2 provided an interface Preparable, which has one method:
    public void prepare();

I made the action implement Preparable interface and moved the above code to inside prepare method. It worked properly.

My action also contains a few other method for different actions, not all of them will need populate this list. I made the method prepare empty, and move the code to new methods named as prepare(). For example, the list is needed in sort(), and search(), but not delete(). I created new methods for them:

    prepareSort() { // will run before sort() ...
    prepareSearch() { // will run before search() ... 
The prepare methods are invoked by the interceptor below:
    <interceptor-ref name="prepare"/>
So, you'll need to make sure it is on the interceptor stack and it appears before interceptor "validation".
    <interceptor-ref name="prepare"/>
    ... 
    <interceptor-ref name="validation"/> 
The preparable interceptor supports a parameter: alwaysInvokePrepare. By default it's true, meaning the prepare methods will run.

There are more details can be found in http://struts.apache.org/2.2.1/docs/prepare-interceptor.html, and this link shows an additional solution: http://struts.apache.org/2.2.1/docs/how-do-we-repopulate-controls-when-validation-fails.html.

Thursday, July 22, 2010

Interval Partitions in Oracle 11g

Among many other enhancements in Oracle 11g, interval partition is definitely a good one for DBAs. There is a good article here discussing it (and other partition enhancements).

In one of our projects, we are looking for sub-partitions within an interval partition, and it's well able to handle. Here's an extended discussion of the link above.

This time we have a interval partition, and list subpartitions inside.
CREATE TABLE interval_tab (
  id           NUMBER,
  code         VARCHAR2(10),
  description  VARCHAR2(50),
  created_date DATE
)
PARTITION BY RANGE (created_date)
INTERVAL (NUMTOYMINTERVAL(1,'MONTH'))
SUBPARTITION BY LIST( code )
SUBPARTITION TEMPLATE
  ( SUBPARTITION CD_01 VALUES ('ONE'),
    SUBPARTITION CD_02 VALUES ('TWO'),
    SUBPARTITION CD_03 VALUES ('THREE')
  )
(
   PARTITION part_01 values LESS THAN (TO_DATE('01-NOV-2007','DD-MON-YYYY'))
);

INSERT INTO interval_tab VALUES (1, 'ONE', 'One', TO_DATE('16-OCT-2007', 'DD-MON-YYYY'));
INSERT INTO interval_tab VALUES (2, 'TWO', 'Two', TO_DATE('31-OCT-2007', 'DD-MON-YYYY'));
COMMIT;
EXEC DBMS_STATS.gather_table_stats(USER, 'INTERVAL_TAB');

COLUMN partition_name FORMAT A20
COLUMN subpartition_name FORMAT A20
COLUMN high_value FORMAT A10

SELECT partition_name, subpartition_name, high_value, num_rows
FROM   user_tab_subpartitions
where table_name = 'INTERVAL_TAB'
ORDER BY table_name, partition_name, subpartition_name;

PARTITION_NAME       SUBPARTITION_NAME    HIGH_VALUE   NUM_ROWS
-------------------- -------------------- ---------- ----------
PART_01              PART_01_CD_01        'ONE'               1
PART_01              PART_01_CD_02        'TWO'               1
PART_01              PART_01_CD_03        'THREE'             0

By adding more data that expands the partitions, you'll see the subpartitions are generated as well.
INSERT INTO interval_tab VALUES (3, 'THREE', 'Three', TO_DATE('01-NOV-2007', 'DD-MON-YYYY'));
INSERT INTO interval_tab VALUES (4, 'TWO', 'TWO', TO_DATE('30-NOV-2007', 'DD-MON-YYYY'));
COMMIT;
EXEC DBMS_STATS.gather_table_stats(USER, 'INTERVAL_TAB');

SELECT partition_name, subpartition_name, high_value, num_rows
FROM   user_tab_subpartitions
where table_name = 'INTERVAL_TAB'
ORDER BY table_name, partition_name, subpartition_name;

PARTITION_NAME       SUBPARTITION_NAME    HIGH_VALUE   NUM_ROWS
-------------------- -------------------- ---------- ----------
PART_01              PART_01_CD_01        'ONE'               1
PART_01              PART_01_CD_02        'TWO'               1
PART_01              PART_01_CD_03        'THREE'             0
SYS_P40              SYS_SUBP37           'ONE'               0
SYS_P40              SYS_SUBP38           'TWO'               1
SYS_P40              SYS_SUBP39           'THREE'             1

Subpartition names are automatically with system name, which is not I'm expecting: it's better to named as partition + subpartition_template, ie, SYS_P40_CD_01.

Friday, April 10, 2009

Encrypt in WebLogic

A colleague gave me a piece of code to plug into my WebLogic 10 JDBC configure file:
<jdbc-data-source ...
...
<password-encrypted>{3DES}xxxxxxxxxxxxxxx</password-encrypted>
...
</jdbc-data-source>

When I copied over, and tried to start WebLogic, got an exception:

weblogic.management.ManagementRuntimeException: com.rsa.jsafe.JSAFE_PaddingException: Could not perform unpadding: invalid pad byte.

It's really not telling what you'll have to do. After some research, I figured out I need to re-encrypt the password. Here's the utility to use:

java -cp <weblogic_home>\server\lib\weblogic.jar -Dweblogic.RootDirectory=<your_domain_dir> weblogic.security.Encrypt <password>

Pasted the result to replace old password, it worked fine.