Apache Camel and WebLogic JMS

Following my last post about my experiences with Apache Camel I wanted to try something a little more advanced. I decided that this time I would add my incoming soap message to a WebLogic JMS Queue.

Creating a JMS queue in WebLogic is beyond the scope of this post. But there should be plenty of guides out there for you to look at. Basically you just need to create a JMS Connection Factory and a JMS Queue attached to the same SubDeployment.

As always, for this series of posts, you can find my example source code on Github here

Maven Dependencies

In addition to the dependencies need in my last example we need to add the camel-jms package.

<dependency>
<artifactId>camel-jms</artifactId>
<groupId>org.apache.camel</groupId>
<version>2.18.0</version>
</dependency>

This will allow us to work with JMS queues.

Spring Configuration

In addition to the configuration of Spring in the last example, we need to add a new section to our spring-ws-servlet.xml file.

<!-- Connection Factories -->
<bean id="weblogic" class="org.apache.camel.component.jms.JmsComponent">
<property name="connectionFactory" ref="orderConnectionFactory"/>
</bean>
<jee:jndi-lookup id="orderConnectionFactory" jndi-name="jms/OrderConnectionFactory"/>

This will tell Spring about our WebLogic JMS Connection Factory so it can be used in our code. The observant reader might notice that we added a new xml prefix (jee) which we of course need to add to our namespace declarations so that our root element now look like this

<beans xmlns="http://www.springframework.org/schema/beans"
 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
 xmlns:camel="http://camel.apache.org/schema/spring"
 xmlns:jee="http://www.springframework.org/schema/jee"
 xsi:schemaLocation="
http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans.xsd
http://camel.apache.org/schema/spring
http://camel.apache.org/schema/spring/camel-spring.xsd
http://www.springframework.org/schema/jee
http://www.springframework.org/schema/jee/spring-jee-2.5.xsd">

Camel Route

In our Camel Route we can now reference our JMS queue with something like this

package dk.moerks.wlsjms.routes;

import dk.moerks.wlsjms.schemas.OrderRequest;
import org.apache.camel.ExchangePattern;
import org.apache.camel.builder.RouteBuilder;
import org.apache.camel.model.dataformat.JaxbDataFormat;

public class OrderRoute extends RouteBuilder {
public void configure() throws Exception {
JaxbDataFormat jaxb = new JaxbDataFormat();
jaxb.setContextPath(OrderRequest.class.getPackage().getName());

from("spring-ws:rootqname:{http://moerks.dk/wlsjms/schemas}OrderRequest?endpointMapping=#endpointMapping")
.setExchangePattern(ExchangePattern.InOnly)
.to("jms:queue:./OrderSystemModule!OrderQueue");
}
}

This will take the incoming soap message and add it to our "OrderQueue". There are a few important notes to be made here though. First you will notice that I used a . in front of the Queue Name. This tells WebLogic to use local lookup on the server connected to. Next you will notice that we have added our System Module to the name. This is because of how WebLogic names the JMS queues internally. The name can be found by clicking on the Queue in the console. It follows the rule <SystemModuleName>!<QueueName> though, so it should be fairly easy to deduct.

That is it. For a full example you can look at my example on Github here

SOAP Service with WebLogic 12c and Camel

I recently acknowledged that I need to expand on my skills. Not only for business reasons, but also to keep me interested in Integration development. I decided that I wanted to focus on Apache Camel first and coming from the Oracle world I thought it would be fun to get a little experience on using Apache Camel with WebLogic. Most of this, first post, isn't WebLogic specific in any way so most of the information in this post should work in pretty much any servlet container. Later on I want to experiment with using WebLogic JMS queues and so on. So stay tuned for more posts in the future.

For my first endeavor I wanted to try and replicate something simple from Oracle Service Bus. So I decided to try to expose a simple SOAP service using Camel to handle the request and response. After searching a bit around I decided to go ahead and use camel-spring-ws for implementing the SOAP endpoint.

My full source is available on GitHub here

In this post I will try to highlight the things that I had trouble with. But this won't be a full tutorial on how to get this working. Think of it more like a collection of notes from my endeavor.

Maven Configuration

To build everything with Maven I used the following dependencies

<dependencies>
<dependency>
<artifactId>camel-core</artifactId>
<groupId>org.apache.camel</groupId>
<version>2.18.0</version>
</dependency>
<dependency>
<artifactId>camel-spring-ws</artifactId>
<groupId>org.apache.camel</groupId>
<version>2.18.0</version>
</dependency>
<dependency>
<artifactId>camel-jaxb</artifactId>
<groupId>org.apache.camel</groupId>
<version>2.18.0</version>
</dependency>
</dependencies>

To enable jaxb generation of java classes from my schemas I needed to add the following plugin

<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>jaxb2-maven-plugin</artifactId>
<version>1.6</version>
<executions>
<execution>
<id>xjc</id>
<goals>
<goal>xjc</goal>
</goals>
</execution>
</executions>
<configuration>
<schemaDirectory>${project.basedir}/src/main/resources/</schemaDirectory>
<outputDirectory>${project.basedir}/src/main/java</outputDirectory>
<clearOutputDir>false</clearOutputDir>
</configuration>
</plugin>

Spring WS Configuration

First thing I needed to do was to get Spring WS to expose my service. This was actually pretty easy to do. In my web.xml file I added the following

<web-app version="2.4" xmlns="http://java.sun.com/xml/ns/j2ee"
 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
 xsi:schemaLocation="http://java.sun.com/xml/ns/j2ee http://java.sun.com/xml/ns/j2ee/web-app_2_4.xsd">

<display-name>Camel and Spring WS</display-name>

<servlet>
<servlet-name>spring-ws</servlet-name>
<servlet-class>org.springframework.ws.transport.http.MessageDispatcherServlet</servlet-class>
<init-param>
<param-name>transformWsdlLocations</param-name>
<param-value>true</param-value>
</init-param>
<load-on-startup>1</load-on-startup>
</servlet>
<servlet-mapping>
<servlet-name>spring-ws</servlet-name>
<url-pattern>/*</url-pattern>
</servlet-mapping>
</web-app>

This adds the Spring WS MessageDispatcherServlet to my web application. To configure this I needed another xml file called spring-ws-servlet.xml with the following content.

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
 xmlns:camel="http://camel.apache.org/schema/spring"
 xsi:schemaLocation="
http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans.xsd
http://camel.apache.org/schema/spring
http://camel.apache.org/schema/spring/camel-spring.xsd">
<bean id="weather" class="org.springframework.ws.wsdl.wsdl11.DefaultWsdl11Definition">
<property name="schema">
<bean id="xsd" class="org.springframework.xml.xsd.SimpleXsdSchema">
<property name="xsd" value="WEB-INF/classes/weather.xsd"/>
</bean>
</property>
<property name="portTypeName" value="weather"/>
<property name="locationUri" value="/weather"/>
</bean>
</beans>

This tells Spring WS to expose my service using the weather.xsd schema. A WSDL file is now available at http://localhost:7001/WEBLOGICCONTEXTROOT/weather/weather.wsdl 

NB! Both of these files should be placed in src/main/webapp/WEB-INF/

Camel Configuration

Now I wanted to have my exposed service pass on the payload to Camel for processing. It turns out that all I needed was to add a few sections to the spring-ws-servlet.xml file we created a minute ago. I added the following sections

<camel:camelContext id="camelContext">
<camel:routeBuilder ref="weatherRoute"/>
</camel:camelContext>
<bean id="weatherRoute" class="dk.moerks.weather.routes.WeatherRoute"/>

<bean id="endpointMapping" class="org.apache.camel.component.spring.ws.bean.CamelEndpointMapping" />

The first adds a Route to Camel and the second simply allows camel to map endpoints to the spring-ws services. After this it is now possible in my Route to reference my Spring WS service.

Camel Route

As seen above I added a Route to Camel called weatherRoute which I implemented in the java class WeatherRoute shown in the following.

package dk.moerks.weather.routes;

import dk.moerks.weather.processors.WeatherProcessor;
import dk.moerks.weather.schemas.WeatherRequest;
import org.apache.camel.builder.RouteBuilder;
import org.apache.camel.model.dataformat.JaxbDataFormat;

public class WeatherRoute extends RouteBuilder{
public void configure() throws Exception {
JaxbDataFormat jaxb = new JaxbDataFormat();
jaxb.setContextPath(WeatherRequest.class.getPackage().getName());

WeatherProcessor weatherProcessor = new WeatherProcessor();

from("spring-ws:rootqname:{http://moerks.dk/soapws/schemas}WeatherRequest?endpointMapping=#endpointMapping")
.unmarshal(jaxb)
.process(weatherProcessor)
.marshal(jaxb);
}
}

This is a simple Camel route that gets the requests from my Spring WS service where the Root Element Name is {http://moerks.dk/soapws/schemas}WeatherRequest. It then unmarshals the request using jaxb and passes it along to a Camel Processor. The result from that Processor is then marshalled using jaxb and passed back to the requesting client.

That is basically it! Of course I wrote a few more files to tie it all together and implemented a stupid Processor, but all of that is available on Github here

Installing WLS 12.2.1.2 in a Docker Container

I started playing around with running WebLogic in docker containers to be able to easily spin up a WebLogic server for simple prototyping etc.

I'll try to sum up my procedure here. Please note there are probably better solution out there, including more official images from Oracle, but something like this works for my use case. Please note that this should only serve as a simple example of how you COULD do it.

Creating the necessary configuration files

Create Dockerfile

Start by creating a Dockerfile with something like the following contents.

FROM centos:7

MAINTAINER Jesper Fussing Mørk <jfm@moerks.dk>

#SET UP ENVIRONMENT
ENV FMW_PKG=fmw_12.2.1.2.0_wls.jar \
JDK_PKG=jdk-8u111-linux-x64.tar.gz \
ORACLE_HOME=/u01/oracle \
JAVA_HOME=/u01/jdk1.8.0_111

#COPY FILES
COPY $JDK_PKG /u01/
COPY $FMW_PKG /u01/
COPY install.file /u01/
COPY oraInst.loc /u01/
COPY template.jar /u01/
COPY create-domain.py /u01/
COPY imageStart.sh /u01/

#RUN INSTALLATIONS
RUN tar zxf /u01/$JDK_PKG -C /u01/

RUN chmod a+xr /u01 && \
useradd -b /u01 -M -s /bin/bash oracle && \
chown oracle:oracle -R /u01 && \
echo oracle:oracle | chpasswd

RUN su -c "$JAVA_HOME/bin/java -jar /u01/$FMW_PKG -silent -responseFile /u01/install.file -invPtrLoc /u01/oraInst.loc -jreLoc $JAVA_HOME -ignoreSysPrereqs -force -novalidation ORACLE_HOME=$ORACLE_HOME INSTALL_TYPE=\"WebLogic Server\"" - oracle

RUN rm /u01/$JDK_PKG /u01/$FMW_PKG /u01/oraInst.loc /u01/install.file

#Start WebLogic Server
WORKDIR $ORACLE_HOME
CMD su -c /u01/imageStart.sh - oracle

This will create a CentOS container and install the WebLogic Server software. It will also create a simple domain and start the WebLogic Server.

Create install.file

Create a text file in the same folder with the name install.file with the following contents:

# Copyright (c) 2014-2015 Oracle and/or its affiliates. All rights reserved.
[ENGINE]

#DO NOT CHANGE THIS.
Response File Version=1.0.0.0.0

[GENERIC]

DECLINE_SECURITY_UPDATES=true
SECURITY_UPDATES_VIA_MYORACLESUPPORT=false

Create oraInst.loc

Create another text file with the following contents:

inventory_loc=/u01/oracle/.inventory
inst_group=oracle

Create imageStart.sh

To prepare and start the WebLogic server when we run the image we need a script that starts it. Something like the following should do it

#!/bin/sh

export MW_HOME=/u01/oracle

/u01/oracle/oracle_common/common/bin/wlst.sh /u01/create-domain.py

/u01/oracle/user_projects/domains/poc/startWebLogic.sh

Create create-domain.py

We also need a WLST script that will create our domain from a domain template. The following is a very simple example

MIDDLEWARE_HOME = os.getenv("MW_HOME")

readTemplate("/u01/template.jar")

cd('/Security/poc/User/weblogic')
cmo.setPassword('Manager1')

setOption('OverwriteDomain', 'true')
print MIDDLEWARE_HOME+'/user_projects/domains/poc'
writeDomain(MIDDLEWARE_HOME+'/user_projects/domains/poc')
closeTemplate()
exit()

Exactly how to create a domain template is out of scope for this little guide, But it is as simple as creating domain and running the config_builder tool.

Download the needed Software

Download the WebLogic generic server i and JDK 8  tar.gz file from OTN

Unzip the WebLogic Server zip file and copy the jar file and the JDK tar.gz file to the folder where you created your Dockerfile.

Build the Image

Run the following command in the folder where you created your Dockerfile

docker build -f Dockerfile -t oracle/wls:12.2.1.2 .

If everything goes well the Image should be build. You might have to adjust some of the environment variables to your specific downloaded files.

Running the image

To actually run the image, issue the following command

docker run -ti -p 7001:7001 oracle/wls:12.2.1.2

This should start everything up and the weblogic server should now be available on port 7001

Recovering from Composite Deployment Failure

I recently had to recover a SOA suite server after something went wrong with one of the composite deployments. During startup of the server it would show in the diagnostic logs that there was a problem finding the composite.xml of one of the composites. That resulted in the soa-infra deployment being unable to start (Really Oracle!!? Why would this be a terminal failure?)

Anyway, luckily there is a way of deleting that particular composite from the MDS. I found the solution in this note on Oracle Support: 1380835.1

 Once you've obtained the EAR file from the support note, the steps are pretty simple:

  1. Download and copy the ShareSoaInfraPartition.ear file to $MIDDLEWARE_HOME/oracle_common/common/bin
  2. cd to $MIDDLEWARE_HOME/oracle_common/common/bin
  3. run wlst.sh
  4. connect to AdminServer ex. connect('weblogic','password','t3://localhost:7001')
  5. run deploy('ShareSoaInfraPartition','ShareSoaInfraPartition.ear',upload='true')
  6. run exportMetadata(application='ShareSoaInfraPartition',server='AdminServer',toLocation='/fmw11g/fmw1115/Middleware',docs='/deployed-composites/deployed-composites.xml')
  7. Updated the deployed-composites.xml file and delete the offensive composite from the file.
  8. run importMetadata(application='ShareSoaInfraPartition',server='AdminServer',fromLocation='/fmw11g/fmw1115/Middleware',docs='/deployed-composites/deployed-composites.xml')
  9. Disconnect from the server ex. disconnect()
  10. Restart your SOA servers

Adding AQ Adapter configuration using WLST

Having to deploy a service that uses the AQ Adapter on multiple servers I set out to create a WLST script that could create the necessary configuration from a property file. It took me a little while to get everything working as I wanted it to. Once you figure it all out, it really isn't that difficult, but there weren't a lot of guies specifically for AQ  Adapter configuration. There are plenty for the DB Adapter and the idea is basically the same.
 

Creating AQ Adapter configuration

I'll post the script here to serve as an example. It might take a little adapting to use it in your own environment.

 

 

from java.io import FileInputStream

#==============================

# Main Method

#==============================


def main(argv):

if (len(argv) <= 3):

usage();

sys.exit(-1);


# Variables

local = argv[1]

wls_username = argv[4]

wls_password = argv[5]

wls_host = argv[6]

wls_port = argv[7]

mw_home = argv[8]

domain_name = argv[2]

userConfigFilePath='c:\\Oracle\\Domains\\'+domain_name+'\\bin\\custom\\user'

userKeyFilePath='c:\\Oracle\\Domains\\'+domain_name+'\\bin\\custom\\key'


#Connect to Admin Server

if local == 'true':

 

 connect(wls_username,wls_password,url='t3://'+wls_host+':'+wls_port)

else:

 domain_home='c:\\Oracle\\Domains\\'+domain_name

 connect(userConfigFile=userConfigFilePath, userKeyFile=userKeyFilePath, url='t3://'+wls_host+':'+wls_port)


properties = getProperties(argv[3]);


print "Creating AQ configuration"

createAQ(properties, mw_home)

print "AQ configuration updated!"


def createAQ(properties, mw_home):

#Global Variables

aqAdapterName = 'AqAdapter'

aqAdapterPath = mw_home + '/Oracle_SOA1/soa/connectors/AqAdapter.rar'


try:

edit()

startEdit()


#Load Application

print "Loading Application"

currentPlanAdapterPath = get('/AppDeployments/AqAdapter/PlanPath')

aqAdapter = loadApplication(aqAdapterPath, currentPlanAdapterPath)

#Configure Properties

print "Creating Variables"

print "Creating ConnectionInstance variable"

aqConnectionInstanceValue = properties.get('aq.connectioninstance.value')

aqConnectionInstanceName = 'ConnectionInstance_' + aqConnectionInstanceValue

aqConnectionInstanceXPath = '/weblogic-connector/outbound-resource-adapter/connection-definition-group/[connection-factory-interface="javax.resource.cci.ConnectionFactory"]/connection-instance/[jndi-name="' + aqConnectionInstanceValue + '"]/jndi-name'

createAppVariable(aqAdapter, aqConnectionInstanceName, aqConnectionInstanceValue, aqConnectionInstanceXPath)


print "Creating DataSource variable"

aqDataSourceName = 'ConfigProperty_DataSourceName_' + aqConnectionInstanceValue

aqDataSourceValue = properties.get('aq.datasource.value')

aqDataSourceXPath = '/weblogic-connector/outbound-resource-adapter/connection-definition-group/[connection-factory-interface="javax.resource.cci.ConnectionFactory"]/connection-instance/[jndi-name="' + aqConnectionInstanceValue + '"]/connection-properties/properties/property/[name="DataSourceName"]/value'

createAppVariable(aqAdapter, aqDataSourceName, aqDataSourceValue, aqDataSourceXPath)


#Save Application

aqAdapter.save()


save()

activate()

redeploy(aqAdapterName, currentPlanAdapterPath, targets=cmo.getTargets());

except Exception, e:

print 'Failed to create AQ configuration'

stopEdit('y')

sys.exit(1)


def createAppVariable(wlstPlan, name, value, xpath, overrideName='AqAdapter.rar', moduleDescriptorName='META-INF/weblogic-ra.xml', origin='planbased'): 

#Ensure variable is overwritten

wlstPlan.destroyVariable(name)

wlstPlan.destroyVariableAssignment(name, overrideName, moduleDescriptorName)

#Create Variable Assignment

variableAssignment = wlstPlan.createVariableAssignment(name, overrideName, moduleDescriptorName)

variableAssignment.setXpath(xpath)

variableAssignment.setOrigin(origin)


#Assign Value to Variable

wlstPlan.createVariable(name, value)


#==============================

# Helper Methods

#==============================


def getProperties(file):

try:

prop = FileInputStream(file)

properties = Properties()

properties.load(prop)

except Exception, e:

print "Something fishy about that file <"+file+">"

sys.exit(0)

return properties



def usage():

print 'Usage: '+sys.argv[0]+' <local> <domain_name> <propertyfile> <weblogic username> <weblogic password> <weblogic host> <weblogic port> <mw_home>'


#==============================

# Start Main

#==============================

main(sys.argv)

This should get you going, and hopefully save someone some hassle of getting this going themselves.

SOA Suite 11g Unit Testing Experiences

I've been working on doing some unit testing for our SOA composites over the last few days. It has been one of those things that we keep putting off, but I finally got around to it and I wanted to write down some of the things I learned over the last few days.

Key Take Aways

You need a SOA Suite server

One of the first things that I was thoroughly surprised by was that you actually need a full running SOA Suite server to be able to do any sort of Unit Testing on your composite. Bad Oracle, Bad Oracle... I understand that the engines that the various components go through (BPEL, Mediator etc) are needed, but there should be a more lightweight alternative to having a full running Suite.

There are Ant scripts available

I kind of figured there were Ant Scripts available, given that we have the whole ant-sca package of Ant scripts to use for building, deploying etc. The one you want to look at in this case is the ant-sca-test.xml Ant script. It has two targets. A "test" and a "report" target. It wasn't immediately apparent to me how they worked, even after reading the documentation. But with a little trial and error I managed to incorporate it into our normal build scripts using the following:

    <ant antfile="${ant-sca-test}" target="test" inheritall="false">
      <!-- Environment Properties -->
<property name="scatest.result" value="${build}/test" />
<property name="scatest.input" value="${project.name}"/>
<property name="scatest.partition" value="default"/>
<property name="scatest.format" value="junit"/>
    <property name="jndi.properties.input" value="${build}/jndi.properties"/>
    </ant>

    <ant antfile="${ant-sca-test}" target="report" inheritall="false">
      <!-- Environment Properties -->
<property name="scatest.result" value="${build}/test" />
    </ant>

From these ant calls we can see that I call the "test" target first before calling the "report" target. The important bit that took me awhile to figure out was which properties I needed to set.
  • scatest.result
    • The folder where the test results are stored. It's important that this value is the same for both targets
  • scatest.input
    • The Name of the Composite to test. This was not clear to me from the beginning. I initially thought it was the path to the composite source.
  • scatest.partition
    • The partition the composite is deployed to
  • scatest.format
    • I'm not sure which values are available here, but I do know that the report target doesn't work if it isn't set to junit
  • jndi.properties.input
    • A property file with properties describing the server. I'll elaborate on that below.
The jndi property file needs the following values:
java.naming.factory.initial=weblogic.jndi.WLInitialContextFactory
java.naming.provider.url=t3://SOA_SERVER_HOST:SOA_SERVER_PORT/soa-infra
java.naming.security.principal=WEBLOGIC_USERNAME
java.naming.security.credentials=WEBLOGIC_PASSWORD
dedicated.connection=true
dedicated.rmicontext=true

The values should be self explanatory.

Final thoughts

With these things sorted out we can now run "ant test" in our build scripts and it will build, deploy and run the unit tests against our server. I still feel that it is a bit crazy that we need a full SOA Suite installation to run these tests, but I guess it's as good as it's going to get at the moment. It also seems it has been improved a lot in the 12c version of the Oracle SOA suite. As we get more into this I will try to either update this post, or write new ones.

How to Create a PKIProvider using WLST

When do you need this

You need to configure at PKI provider if you want to use Service Key Providers in Oracle Service Bus. When trying to use a Service Key Provider in the OSB, yo might have run into the following error:

There is no PKI credential mapper provider configured in your security realm. Service key provider management will be disabled. Configure a PKI credential mapper provider if you need service provider support. This is typically the case if you have Oracle Service Bus proxy services with web service security enabled or outbound 2-way SSL connections.

Fortunately it is quite easy to create a PKI Provider using the WebLogic Console. But sometimes it's just easier to do it using WLST, especially if you have many environments to configure.

Creating a PKI Provider with WLST

Fortunately it is quite easy to create a PKI provider using WLST as well. The following WLST script should give you an idea of how to create a PKI Provider

#Fetch domain name from arguments
if len(sys.argv) == 4:
domain_name = sys.argv[1]
filepath = sys.argv[2]
password = sys.argv[3]
else:
  print 'Usage: wlst.cmd change-ds-password.py <DOMAIN_NAME> <KEYSTORE FILEPATH> <KEYSTORE PASS>'

# Variables
domain_home=/<PATH TO DOMAINS FOLDER>/+domain_name

#Connect to Admin Server
connect('weblogic','<WLS PASSWORD>','t3://localhost:7001')

edit()
startEdit()

cd('/SecurityConfiguration/'+domain_name+'/Realms/myrealm')
cmo.createCredentialMapper('PKIProvider', 'weblogic.security.providers.credentials.PKICredentialMapper')

cd('/SecurityConfiguration/'+domain_name+'/Realms/myrealm/CredentialMappers/PKIProvider')
cmo.setKeyStoreFileName(filepath)
encrypted_password = encrypt(password, domain_home)
cmo.setKeyStorePassPhraseEncrypted(encrypted_password)

save()
activate()
disconnect()

Of course you would need to replace PATH TO DOMAINS FOLDER and WLS PASSWORD with proper values for you environment.

How To configure SOA coherence clustering using WLST

When building a SOA cluster using WLST you still have to configure the Coherence clustering in the WebLogic startup parameters. I'll explain how this can be done using WLST in the following.

WebLogic has two methods of setting startup parameters. You can either change the scripts used for starting the server, or you can set them in a textfield in the server options.

We are going to do something similar to the latter method. But instead of using the WebLogic console, we will do this using WLST.

WLST can manipulate a domain either Offline or Online. Online means connecting to the AdminServer and running WLST against that. The other way is to read the Domain Configuration from the file system and manipulate it before writing it back to the file system. I will show you how to do both.

Online

To configure the coherence cluster using Online WLST you would do this:

connect(ADMIN_USER, ADMIN_PASS, ADMIN_ADDR)
edit()
startEdit()

#Build general arguments
arguments = []
arguments.append('-Dtangosol.coherence.wka1=<SOA HOST 1> -Dtangosol.coherence.wka1.port=9088 ')
arguments.append('-Dtangosol.coherence.wka2=<SOA HOST 2> -Dtangosol.coherence.wka2.port=9088 ')
general_arguments = ''.join(arguments)

#Build Local Arguments
local_arguments = general_arguments + '-Dtangosol.coherence.localhost=<SOA HOST 1> -Dtangosol.coherence.localport=9088
cd('Servers/<SOA SERVER 2>')
cmo.getServerStart().setArguments(local_arguments)
local_arguments = general_arguments + '-Dtangosol.coherence.localhost=<SOA HOST 2> -Dtangosol.coherence.localport=9088
cd('Servers/<SOA SERVER 2>')
cmo.getServerStart().setArguments(local_arguments)
save()

Offline

To configure the coherence cluster using Offline WLST a few modifications to the above needs to be done:

readDomain('<PATH TO DOMAIN>')

#Build general arguments
arguments = []
arguments.append('-Dtangosol.coherence.wka1=<SOA HOST 1> -Dtangosol.coherence.wka1.port=9088 ')
arguments.append('-Dtangosol.coherence.wka2=<SOA HOST 2> -Dtangosol.coherence.wka2.port=9088 ')
general_arguments = ''.join(arguments)

#Build Local Arguments
local_arguments = general_arguments + '-Dtangosol.coherence.localhost=<SOA HOST 1> -Dtangosol.coherence.localport=9088
cd('/Servers/<SOA SERVER 1>')
create('<SOA SERVER 1>', 'ServerStart')
cd('ServerStart/<SOA SERVER 1>')
set("Arguments", arguments)
local_arguments = general_arguments + '-Dtangosol.coherence.localhost=<SOA HOST 2> -Dtangosol.coherence.localport=9088
cd('/Servers/<SOA SERVER 1>')
create('<SOA SERVER 1>', 'ServerStart')
cd('ServerStart/<SOA SERVER 1>')
set("Arguments", arguments)
save()
setOption('OverwriteDomain', 'true')
writeDomain('<PATH TO DOMAIN>')
exit()

After everything is said and done you just have to restart the servers and the coherence cluster should be running in unicast clustering mode.

NB: The above code is untestet, but you should be able to get the general idea.

Network issues with Coherence

I recently had to enable the normal unicast Coherence cluster required by the Oracle SOA suite on a set of Windows servers which was completely separated by local firewalls. It took a good bit of trial and error to get it to work, so I thought I would sum up what I learned during the endevaour.

Error Messages

Initially we saw a lot of deployment and Coherence errors with timeouts and connection issues like these:

[oracle.integration.platform.blocks.cluster.CoherenceClusterInterfaceImpl]: Constructor threw exception; nested exception is com.tangosol.net.RequestTimeoutException: Timeout during service start: ServiceInfo(Id=0, Name=Cluster, Type=Cluster......

[2014-10-08T13:22:14.221+02:00] [soa_server02] [WARNING] [] [Coherence] [tid: Logger@1932191207 3.7.1.1] [userId: <anonymous>] [ecid: 0000KZjU_CZEKQG5Mzt1ie1KDHoj000002,0] [APP: soa-infra] 2014-10-08 13:22:14.221/375.070 Oracle Coherence GE 3.7.1.1 <Warning> (thread=Cluster, member=n/a): Delaying formation of a new cluster; IpMontor failed to verify the reachability of senior Member(Id=1, Timestamp=2014-10-08 13:18:54.344, Address=10.0.0.1:8088, MachineId=3060, Location=site:,machine:exampleerver1,process:4864, Role=WeblogicServer); if this persists it is likely the result of a local or remote firewall rule blocking either ICMP pings, or connections to TCP port 7

<08-10-2014 15:48:30 CEST> <Error> <Coherence> <BEA-000000> <2014-10-08 15:48:30.468/517.608 Oracle Coherence GE 3.7.1.1 <Error> (thread=Cluster, member=n/a): Detected soft timeout) of {WrapperGuardable Guard{Daemon=IpMonitor} Service=ClusterService{Name=Cluster, State=(SERVICE_STARTED, STATE_JOINED), Id=0, Version=3.7.1, OldestMemberId=1}}> 
<08-10-2014 15:48:30 CEST> <Error> <Coherence> <BEA-000000> <2014-10-08 15:48:30.499/517.639 Oracle Coherence GE 3.7.1.1 <Error> (thread=Recovery Thread, member=n/a): Full Thread Dump

<08-10-2014 15:48:56 CEST> <Error> <oracle.integration.platform.blocks.deploy.coordinator> <BEA-000000> <Cluster communication initialization failed.  If you are using multicast, please make sure multicast is enabled on your network and that there is no interference on the address in use.  Please see the documentation for more details.
com.tangosol.net.RequestTimeoutException: Timeout during service start: ServiceInfo(Id=0, Name=Cluster, Type=Cluster
  MemberSet=MasterMemberSet(.......

Needed Ports

During the normal configuration of Coherence you define, among other things, what port it should run on. What isn't apparent is that Coherence actually uses two ports. The one you configure + The one above. So if you configure it to use 8088, it also needs 8089. Ensure that both ports are open on TCP and UDP both. Besides that, the firewall needs to allow communication on port 7 (Echo) and to allow ICMP ping packages. This is used to keep the Coherence cluster alive.

Echo Service

Even though the Coherence documentation states that port 7 just needs to be open. I actually had to have a proper echo service respond. In windows this is done by adding the feature "Simple TCP/IP Services".

It took a bit of research to figure all of this out, so I hope it helps others having problems with Coherence clustering.

Deleting the SOA schemas manually

We recently had to delete the SOA schemas, tables and so on manually on one of our installations. It turned out it wasn't as easy as just dropping the tables space used. After having dropped and recreated the tablespace we wanted to use, the Oracle RCU tool still had problems with using the same prefix we had previously used. After some research, by a DBA, it was discovered that the RCU tool apparently creates a table in the System tablespace called system.SCHEMA_VERSION_REGISTRY$ If we look what is present in that table we se that:

select * from system.SCHEMA_VERSION_REGISTRY$;
MDS Metadata Services
SOA MDS
MDS SOA_MDS
11.1.1.7.0 VALID
N

07-OCT-14 02.24.14.455000 PM

ORASDPM
SDP Messaging
SOA ORASDPM
ORASDPM SOA_ORASDPM
11.1.1.7.0 VALID
N

07-OCT-14 02.24.17.731000 PM

SOAINFRA
SOA Infrastructure Services
SOA SOAINFRA
SOAINFRA SOA_SOAINFRA
11.1.1.7.0 VALID
N

07-OCT-14 02.25.07.011000 PM

BAM
BAM Services
SOA BAM
BAM SOA_ORABAM
11.1.1.5.0 VALID
N

07-OCT-14 02.25.36.480000 PM

So apparently this is where RCU keeps track of which prefixes has been used. After deleting the contents by:
delete from system.SCHEMA_VERSION_REGISTRY$;
We could again use the SOA prefix during the RCU installation.

SOA Suite 12c From a Developer Perspective

The 12c version of the SOA Suite has finally been released. And seen from a developer perspective there are a lot of notable news. I will summarize the ones I've discovered so far but I'm sure there are a lot more packed under the hood that I haven't discovered yet.

Installer

Of course the major new feature from a developer perspective is the new installation procedure. I tried the installer in a Virtual Environment (VirtualBox) on CentOS and everything was installed very easily using the provided quickstart installer. I will try to write a post about the installation process on a later date.

New Features

There are a lot of new features in Oracle SOA Suite 12c. The most notable of those are:
  • Coherence Adapter
    • You can now easily cache data across the entire platform.
  • MSMQ Adapter
    • Something many have been missing in the past. You can now connect to a Microsoft MSMQ directly. This can be very useful when doing integrations with Microsoft based technologies.
  • Cloud Adapter
    • This is the first rendition of the Oracle Cloud Adapter and it initially supports Salesforce.com and from the documentation it will "soon" support a suite of the Oracle Cloud products. However the really interesting part of this adapter is the availability of a Cloud Adapter SDK so you can develop your own Cloud Adapter integrations. This could prove very useful and will hopefully allow third party cloud services to enable integrations with Oracle SOA Suite using the Cloud Adapter.
  • Oracle Service Bus integration
    • The Oracle Service Bus has now been more tightly integrated with the SOA Suite. Development on the Service Bus has now, finally, been integrated with JDeveloper allowing a developer to have a onestop IDE.
  • Templates
    • It seems that a lot of effort has gone into making template support a lot more useful. You can now create templates for a wide range of things. This will allow to have a more generic way of solving common problems in the code.

JDeveloper News

JDeveloper has seen a major overhaul for the 12c release. As written above, Service Bus development has been integrated and there are a lot of other new features which can help in the development. I will list the most notable below.
  • Maven support
    • This is really cool, and something that has been missing for a long time. There is now builtin maven support. This will make integration with Agile development a lot simpler and make it easy to use Hudson/Jenkins and similar products more useful in the development process.
  • Graphical XQuery editor
    • Something I know a lot of developers have been asking about. It seems that doing XQuery in your process have now become easier with the new editor. I haven't tried this yet, but it should prove useful
  • Debugging
    • Debugging your components have also become a lot easier. It is now possible to set beakpoints in your composite which will definetely make it easier to figure out what is going on, on a particular section of your code.
  • Testing
    • Testing has been enhanced which will make unit testing much more useful. This is something that have really been missing in previous releases.
  • Fault Handling
    • Doing fault handling in 11g was quite cumbersome and you had to write XML files by hand. This has now been made easier with a proper Fault Handling editor in 12c.
There are of course a lot of other features in Oracle SOA Suite 12c. But from my short time with the product, the features described above seems like the most important from a developer perspective. At least on a simple development basis. Many of the more advanced features will require more time to start using as they are probably more suited for speciel requirements I don't see that often in "the real world".

How to Pass SOAP headers through a Mediator

In the Oracle SOA Suite, the Mediator component will, by default, strip the SOAP headers from a message. Changing that behaviour is quite simple though and can sometimes be really useful.

Example configuration snippet

To change the behaviour just add the following property to you Mediator components in your composite.xml from:

<component name="MediatorName">
<implementation.mediator src="MediatorName.mplan"/>
</component>

To:
<component name="MediatorName">
<implementation.mediator src="MediatorName.mplan"/>

  <property name="passThroughHeader">true</property> </component>

That's it! Your SOAP headers should now be passed through the mediator.

Oracle WSM and 256bit certificates

I recently had a problem where it wouldn't accept some certificates we had to use to connect to a third party.

The Problem

We saw exceptions like this:

java.security.InvalidKeyException: Illegal key size

in the log files and after a bit of search we found that it all boils down to the standard jvm not supporting certificates above 128bit.

The Solution

The solution was to download the "Java Cryptography Extension (JCE) Unlimited Strength" Policy files and follow the installation instruction in that package.

NOTE: This works for JRockit as well even though it isn't specifically specified in the documentaion.

Oracle WSM

The standard Oracle WSM policies uses Basic 128bit encryption but they can be configured to use other encryption algorithms. Under advanced settings for the impacted policies there is an Algorithm Suite setting that allows you to change to the an algorithm that fits the certificates.

It is probably a good idea to make a copy of the policy before starting to configure it. This can be done using the Create Like button.

Export X509 certificate from PFX keystore with OpenSSL

Sometimes we find ourselves in the need of converting certificates between various formats.

Recently I received a .pfx keystore (pkcs12 format) containing a X509v3 certificate I needed to import in a java jks keystore.

Exporting the certificate from the .pfx file

Exporting the certificate is pretty easy, just use the following command:
openssl pkcs12 -in pfxkeystore.pfx -out certificate.pem -nokeys -nodes

Convert to x509 format

The exported certificate is not a x509 certificate. To convert it use the following command:
openssl x509 -outform der -in certificate.pem -out certificate.cer

Import Certificate into jks using keytool

Now you can just import the certificate normally into your jks keystore using this command:
keytool -importcert -alias certificatealias -file certificate.cer -keystore jkskeystore.jks

Upgrading Oracle BI Enterprise Edition

This is the second part of my recent adventure with upgrading an Oracle BI Enterprise Edition (OBIEE) installation. This is about upgrading the OBIEE software.

Upgrading OBIEE

This post is about upgrading from 11.1.1.5.0 to 11.1.1.6.0. 

Downloading the software

You can obtain the latest release from OTN. At the time of writing this we are on version 11.1.1.6.0 (as a release version)

Upgrading the software

Upgrading the software is actually quite easy, but there is a few steps you need to take care of.

Stopping the environment

Before upgrading the software make sure everything is stopped. This includes WebLogic instances, Process Manager, NodeManager etc.

Patch WebLogic Server

As described in my previous post. Make sure that the WebLogic server has been upgraded first.

Upgrade OBIEE software

When the above has been done, you are ready to run the OBIEE software installer. Just run the installer but make sure to select the "Install software only" option when prompted and point the installer to your current installation folder.

Update the database schemas

After upgrading the software you need to patch the database schemas to the latest version. This is done by running the Patch Set Assistant from the MIDDLEWARE_HOME/oracle_common/bin folder by typing
./psa (or psa.bat if you're on windows)

The following schemas needs to be updated
  • MDS
  • BIPLATFORM
Hint. If you select the BI option, it automatically selects the MDS as well

Troubleshooting

If you get an Error message that says "Invalid JSON String: 0 {t:4}". After the upgrade. Please try the following
  • Clear Browser Cache (This can be rather tricky if you are using IE)
  • Redeploy analytics.ear
Normally clearing the browser cache is enough.

If you get a problem where the system is unresponsive when clicking on surden things in the UI, it is caused by a deployment error. You need to completely remove the analytics.ear deployment using the WLS console.

  • Undeploy analytics
  • Restart the servers
  • Deploy analytics.ear again
I don't know why this is necessary, but that eventually fixed the issue for me.


After upgrading the software to the latest release version you will want to patch it to the latest patch set.

Patching Oracle BI Enterprise Edition

This is the third part of my recent adventure with upgrading an Oracle BI Enterprise Edition (OBIEE) installation. This is about patching the OBIEE to the latest patchset release.

Patching OBIEE

When writing this I was patching from 11.1.1.6.0 to 11.1.1.6.4. As this was the requested patchset by the customer.

Obtaining the Patches

All of the patches can be obtained from Oracle Support. Figuring out which patches you need can be quite tricky. But they can all be found by doing the following.
  1. Click on the "Patches & Updates" tab
  2. In the "Patch Search" box select the following
    1. Product/Family = Oracle BI Suite EE
    2. Release = 11.1.1.6.4
  3. Select the patches and download the files

Applying the Patches

Unzip the patch zip files into the MIDDLEWARE_HOME/Oracle_BI1 folder.

Then following the README files from the first patch, you should set the following environment variables. As this patch was done on windows you would do the following. You should be able to rewrite for other operating systems.

set ORACLE_HOME=e:\Omit\OWL1035\Oracle_BI1
set PATH=%ORACLE_HOME%\bin;%PATH%
set JAVA_HOME=%ORACLE_HOME%\jdk
set PATH=%JAVA_HOME%\bin;%PATH%
set PATH=%ORACLE_HOME%\OPatch;%PATH%

Then for each of the unzippen patch folders. run opatch apply. You should do this in the order specified in the README file from the first patch.

Post Patch tasks

After patching the product you should do the following:
  • Copy the BIPublisherDesktop32.exe and BIPublisherDesktop64.exe files from the ORACLE_BI_HOME/clients/bipublisher/repository/Tools directory to the MW_HOME/user_projects/domains/bifoundation_domain/config/bipublisher/repository/Tools directory.
  • Remove the jsp_servlet subdirectory associated with Oracle BI Publisher (if it exists), as follows:
    • If it exists, the jsp_servlet subdirectory is located in one of the directories below the <Middleware Home>/<Domain Home>/servers/bi_server1/tmp/_WL_user/bipublisher_11.1.1 directory.
  • Start all the BI components (AdminServer, Managed Server and Process Manager)
After having done all of the above you should now have a fully patched OBIEE installation.

Upgrading Oracle WebLogic Server

As part of patching an Oracle BI Entreprise Edition (OBIEE) installation I had to patch an Oracle WebLogic Server (WLS) from version 103.5 to 10.3.6. This post describes the process of doing this update.

Upgrading WLS from 10.3.5 to 10.3.6

Patching a WebLogic server is a fairly easy process. And can be done in the following steps

Download the software

You can download the WebLogic Server software from Oracle Support. It isn't easy but it has to be done.
Once you have logged in to Oracle Support do the following:
  1. Click on the "Patches & Updates" tab
  2. In the "Patch Search" box select the following
    1. Product - Oracle WebLogic Server
    2. Release - 10.3.6 (or whatever release you are looking for)
    3. Platform - Whatever platform you are looking for
    4. Description - upgrade
    5. Click Search
  3. Click the Patch Name/Number
  4. Click the Download button.
This should get you the upgrade installer you need.

Backup your software and domain folders

Just to be on the safe side it is a good idea to backup your software and domain folders. The easiest way to do that is to Zip the folders and copy the zip files to a safe place.

Run the installer

To run the installer simply run the downloaded file. On 64 bit architectures that is most like running:

java -jar wls1036_upgrade_installer.jar

Once the installer is ready it is simply going through the install wizard.

The next part is to patch the OBIEE software with the latest version.

Forcing IP version on Oracle WebLogic

I was recently called out to a customer to fix a networking issue on WebLogic.
The problem was that they had two different instances. Instance one makes a transactional request to an EJB on the second instance. On the second instance it was clear that the transaction was commited, but the first node never got the response. We tracked the issue down to the network level. Server one made the request using one IP version and the second instance responded on another. Apparently this mix of IP versions was the culprit.
The solution, we chose, was to force the JVM for the instances to use the IPv4 protocol. This is done by setting the following startup parameters.

-Djava.net.preferIPv6Addresses=false -Djava.net.preferIPv4Stack=true

Starting WebLogic with those parameters fixed the issue.