DataPower: How to duplicate, rename, and bulk delete objects?

Introduction

Did you know that all the objects that exist in a domain are there just because of a single configuration file? Did you know that you can wipe out all the objects by simply deleting the content of this file? Did you know that you can manipulate all the objects using your favorite text editor?

If you answered yes to any of the above questions, you are done and don't need to spend any more time at this article. If not, take a seat, relax, and enjoy these tips as you become a DataPower cheater.

Understanding the case

While booting, DataPower will look for all the .cfg files spread over all the domains. These .cfg files contain instructions on what objects and configuration should be loaded to the memory for execution time. Actually, the only reason why you have the objects in place when you restart your domain or recycle the entire box is because of these .cfg files that are storing details of almost everything for you.

Take a moment to analyze any .cfg file you may have in the default or any other existing domain. Usually they are named after the domain name, so if you have a domain called Sandbox, you will have one config:///Sandbox.cfg file. The .cfg file of the default domain falls into an exception and will be called autoconfig.cfg.

Take a moment there, start reading some lines, you will soon realize that all the objects you have created using any DataPower interface will be there. Created a new Crypto Profile? It will be there. Created a new Deployment Policy? It will also be there. Defined a password for a local user? Nah, that will not be there, and honestly, I don't know where this information is stored. :-)

Usage

I particularly use this a lot. Duplicating, triplicating and sometimes quadruplicating Deployment Policies used to kill me back in the day. That was because I used to do all that using the WebGUI. If you have already done that you know what I am talking about. Imagine a scenario which you need to create one Deployment Policy for each environment, let's say one for UNIT, one for INTG, one for PERF, and one (or more) for PROD. I would imagine you would keep the one for UNIT that you supposedly created first open in one browser tab and visually refer to it when creating the others. That means a lot of eyes work from left to right and MANY clicks to add the Modified Configurations. So after dying a few times, I decided to investigate a less time consuming way to do this, and that was when I had this idea to do this through the .cfg file.

My UNIT deployment policy was the following:

deployment-policy "Service-UNIT-DeploymentPolicy"

 modify "*/*/wsm/wsm-endpointrewrite?Name=.*&Property=WSEndpointRemoteRewriteRule/RemoteEndpointHostname&Value=.*" "change" "" "unit-server.datapower.com"

 modify "*/*/wsm/wsm-endpointrewrite?Name=.*&Property=WSEndpointRemoteRewriteRule/RemoteEndpointPort&Value=.*" "change" "" "80"

 modify "*/*/services/multiprotocol-gateway?Property=DebugMode" "change" "" "off"

 modify "*/*/services/ws-proxy?Property=DebugMode" "change" "" "off"

exit

So to duplicate something similar for my INTG environment, I just copied it and pasted it on the line below, changing the parameters pertaining to the new environment, such as name, hostname, port, etc:

deployment-policy "Service-UNIT-DeploymentPolicy"

 modify "*/*/wsm/wsm-endpointrewrite?Name=.*&Property=WSEndpointRemoteRewriteRule/RemoteEndpointHostname&Value=.*" "change" "" "unit-server.datapower.com"

 modify "*/*/wsm/wsm-endpointrewrite?Name=.*&Property=WSEndpointRemoteRewriteRule/RemoteEndpointPort&Value=.*" "change" "" "80"

 modify "*/*/services/multiprotocol-gateway?Property=DebugMode" "change" "" "off"

 modify "*/*/services/ws-proxy?Property=DebugMode" "change" "" "off"

exit

deployment-policy "Service-INTG-DeploymentPolicy"

 modify "*/*/wsm/wsm-endpointrewrite?Name=.*&Property=WSEndpointRemoteRewriteRule/RemoteEndpointHostname&Value=.*" "change" "" "intg-server.datapower.com"

 modify "*/*/wsm/wsm-endpointrewrite?Name=.*&Property=WSEndpointRemoteRewriteRule/RemoteEndpointPort&Value=.*" "change" "" "80"

 modify "*/*/services/multiprotocol-gateway?Property=DebugMode" "change" "" "off"

 modify "*/*/services/ws-proxy?Property=DebugMode" "change" "" "off"

exit

Once done, it was just a matter of restarting the domain and the Deployment Policies were replicated.

Other related tips...

Use it to find out the right CLI command to perform a specific action

Don't remember exactly the CLI command to create a crypto object? Just go to the .cfg file and try to find a reference for it, when you find it, that is the command you are looking for. That is because the .cfg is nothing more, nothing less than CLI commands grouped in a single place.

Use it to delete a massive amount of objects

For example, you created a bunch of Web Service Proxies or Multi Protocol Gateway. By default, when you create these objects you are rewarded with a lot of child objects that will not go away even after you remove their parents, such as matches, processing actions, slm policies, etc. Good news is that they are named after their parents, so you can sort them out easily and quickly select them for removal.

Changing object names

In general, you cannot change object names in DataPower. Gave it the wrong name? Delete it and recreate it. As an alternative to this, go to the .cfg file and rename it there. Once done, restart the domain and voilà!

Recommendations

• Be careful. Make sure you know what you are doing before playing with this file. Rule of thumb, always have a backup of it in case something goes wrong.
• Some words are reserved, and the default interfaces won't be there to tell you that, so be careful. Rule of thumb, avoid naming objects that can be confused with an existing command, for example when naming an Alias object, use "xmlAlias" instead of only "xml", as it may be parsed incorrectly by the interpreter when loading the objects to the memory.
• And last but not least, BE CAREFUL, not sure if I have already said that.

Conclusion

The possibilities are endless, you can optimize a lot of your time after mastering on this. Duplicating, bulk deletion, searching for commands, are just the obvious things you can do while playing with the .cfg files. The limit here is your imagination, so if you think of something cool that the knowledge present in this article enabled you to do, don't hesitate to share it with us in the comments section below. Happy .cfg'ing!

DataPower: The Interoperability Test Service

Hate having to create an XML Firewall service configured as loopback every time you want to test a single stylesheet using DataPower extension functions? Well, you may have more options available to you…

Since version 5.0, IBM released a capability called Interoperability Test Service (IOP). One of the features this mode supports is the ability to listen to requests containing a stylesheet file along with its equivalent request, then it processes the instructions present on the stylesheet against its request, and finally sends the output of the stylesheet back as the result.

To use the IOP, you first need to enable the service at Objects > Device Management > Interoperability Test Service.

For the second step, you will need a client to send requests to the service you have just enabled. IBM thinking about on making our lives easier, made available two clients that can be found at Resource Kit 5.0.

After downloading it, just extract it to your local file system, and then refer to the files DPInteropClient.jar or dp-interop-client.sh (depending on your preference, operating, system, etc).

The Resource Kit 5.0 comes with some samples that can be used for testing. The first scenario we will cover here will be the conversion of an XML file into the base 64 by using the dp:encode extension function:

The XML message to be converted to base 64
$ cat message.xml
<msg>Hello World!</msg>

The XSLT used to encode the message into base 64
$ cat toBase64.xsl
<xsl:stylesheet version="1.0" 
  xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
  xmlns:dp="http://www.datapower.com/extensions"
  extension-element-prefixes="dp">

  <xsl:template match="/">
    <xsl:value-of select="dp:encode(., 'base-64')"/>
  </xsl:template>
</xsl:stylesheet>

The IOP client sending both the XSLT and message to be encoded along with the result from DataPower:
$ ../../clients/dp-interop-client.sh -x toBase64.xsl -i message.xml -h [dphostname] -p [port]
>> Creating the XSLT request...
>> Sending the request to http://dphostname:port/
>> No basic authentication is provided.
SGVsbG8gV29ybGQh

Another interesting feature available is the ability to test XPath syntaxes. For example, take into account the following XML (not included in the Resource Kit):

$ cat bookstore.xml 
<?xml version="1.0" encoding="UTF-8"?>
<bookstore>
<book category="COOKING">
<title lang="en">Everyday Italian</title>
<author>Giada De Laurentiis</author>
<year>2005</year>
<price>30.00</price>
</book>
<book category="CHILDREN">
<title lang="en">Harry Potter</title>
<author>J K. Rowling</author>
<year>2005</year>
<price>29.99</price>
</book>
<book category="WEB">
<title lang="en">XQuery Kick Start</title>
<author>James McGovern</author>
<author>Per Bothner</author>
<author>Kurt Cagle</author>
<author>James Linn</author>
<author>Vaidyanathan Nagarajan</author>
<year>2003</year>
<price>49.99</price>
</book>
<book category="WEB">
<title lang="en">Learning XML</title>
<author>Erik T. Ray</author>
<year>2003</year>
<price>39.95</price>
</book>
</bookstore>

Now use the following XPath expression against it (this expression will select the title and price of all books on the category "WEB" that are priced above 39.95:

/bookstore/book[@category=\"WEB\" and price>39.95]/(title|price)

To apply the above XPath expression to the IOP client command, use the following:

$ ../../clients/dp-interop-client.sh -t xpath -e "/bookstore/book[@category=\"WEB\" and price>39.95]/(title|price)" -i bookstore.xml -h [dphostname] -p [port]
>> Creating the XPath request...
>> Sending the request to http://dphostname:port/
>> No basic authentication is provided.
Found 2 nodes:
-- NODE --
<title lang="en">XQuery Kick Start</title>
-- NODE --
<price>49.99</price>

Cool, huh?

Besides XSLT and XPath testing, you can also use this tool to test FFD requests and perform schema validation. For more details, refer to the Interoperability Test Service page at the DataPower Information Center.

I remember that in the past, there used to be a DataPower plugin for Eclipse that enabled you to do practically the same stuff, but for some reason I think it was discontinued by IBM. Have you ever heard of it? Had a chance to use it? How do you compare it with this new method available?

DataPower: IBM technotes are now providing GatewayScript sample code in addition to XSLT

With the recent release of the GatewayScript for the DataPower family of appliances, IBM is now writing technotes that, in addition to XSLT, also contain a sample of code for the new GatewayScript format.

An example of this, would be a technote published last week that describes how to remove the Content-Type header that DataPower automatically adds to the responses. For the XSLT example they provided the following solution:

<xsl:template match="/">
 <dp:remove-http-response-header name="Content-Type"/>
</xsl:template>

Now for the GatewayScript, the following code was provided:

var hm = require('header-metadata');
hm.current.remove('Content-Type');

For experienced DataPower developers, the first solution seems simpler, as it requires only one line of code to perform that action (ignoring of course the opening and closing of the <xsl:template> element).

The GatewayScript solution though, may look simpler for those used to the JavaScript language, which the GatewayScript is based on.

What would have been your choice if you were assigned to solve this issue? XSLT or GatewayScript?

DataPower: Meet the new features that will be available on firmware V7.0

We are exactly one month away from the general availability of the new DataPower firmware V7.0. Although we have already mentioned the capability that will change the course of our lives forever (just kidding), the new firmware will also bring to the table many other cool features that many clients have been asking as enhancements, like the following:

• A dedicated virtual appliance edition for developers
• SFTP support for XG45
• GatewayScript (JavaScript enablement in the processing policy)
• Improved API Management
• Network Link Aggregation for redundancy and increased throughput
• WebSocket Proxy for low-latency communication
• Support for Sterling Multi-Enterprise Integration Gateway (MEIG)
• GTID or Global Transaction ID to ease troubleshooting of chained services
• Citrix XenServer support for additional deployment flexibility

Yesterday IBM released a new slide deck detailing each of the new features. Highly recommended if you/your company have been waiting for any of these features to be available.

DataPower: 5 secrets that you may not have known

Okay, let's go straight to the point, starting with a few questions:

1. Is it possible to disable a Multi-Protocol Gateway or a Web Service Proxy?
2. Is it possible to export a certificate from the cert: directory?
3. Is it possible to increase the number of transactions in the probe history?
4. Can a Front Side Handler object be created automatically when creating a Web Service Proxy?
5. Can a service have higher priority over others?

You might have answered yes to at least one of these questions, but would you be surprised if I told you that the answer for every single one is yes?

Follow the link below to see more details about each of the secrets above.

DataPower: What to expect from the new GatewayScript?

GatewayScript is what is called a new capability that will be available in the upcoming DataPower firmware version 7.0. A lot of attention has been put into this topic recently and it will be present in at least four different presentations at Impact 2014 by the end of this month.

The GatewayScript expands the number of programming languages currently available in DataPower. GatewayScript is completely based on JavaScript language and brings to the table all the power that its base language has to offer.

It does not mean that the available programming languages, such as XSLT and XQuery, were lacking power. The main idea on bringing a new programming language to the table is to leverage common programming skills, speeding up DataPower adoption and making it even more popular.

New ways to debug code have also been made available for the GatewayScript functionality, making it possible to add break points, print variable values, explore stack trace, etc.

Performance is not supposed to be an issue with the new available language, as the code will be compiled, cached and reused on demand. IBM keeps promising wire-speed for processing.

Examples will be provided with the new firmware in order to give us a better idea on what exactly can be accomplished with this new capability. They will be located under store:///gatewayscript/example-*.js.

In the morning of April 15th, Tony Ffrench (STSM DataPower Architect), Tim Smith (DataPower Architect) and Ozair Sheikh (DataPower Product Line Manager), gave a very rich presentation at the WebSphere User Group website, about the new capabilities that will be available on the new firmware version. If you did not have a chance to attend to this webinar, you can download the presentation used by them right here.

To see the official IBM announcement of the new firmware version 7.0, click here.

EDIT: On April 17th, the replay of the above presentation has been made available by the WebSphere User Group website. To watch it, just click here.