Posts Tagged ‘haker’

Rebuilding WebCenter Collaboration Index By Project

Thursday, April 4th, 2013

Another tip of the hat to Brian Hak for this pretty awesome Hak (see what I did there?).

Last year, Brian was faced with a problem: Some documents in WebCenter Interaction Collaboration Server weren’t properly indexed, and his only option was the rebuild the ENTIRE index (a pain we’re all pretty familiar with). With many thousands of documents, the job would never finish, and end users would be frustrated with incomplete results while the job toiled away after wiping everything out.

So he took it upon himself to write CollabSearchRebuilder. CollabSearchRebuilder allows you to rebuild specific subsets of documents without having to wipe out the whole index and start all over again.

Feel free to download the source and build files and check it out!

New Collaboration Server Fix: WebCenter Patch Process?

Sunday, May 13th, 2012

I got a tip a couple months ago (thanks Brian Hak!) about a new Collab patch that “fixes the bulk upload” issue, and yes, it’s taken me this long to actually dig in a bit.

At first, I thought this was the same one I wrote about in February of last year – the IE8 Critical Fix. But, it turns out that this patch was released in May in 2011.

From the release notes in the patch called “Patch 12423316: WLS1034: UNDER JROCKIT, COLLABORATION PROJECT ACCESS FAILS WITH TREEUNMARSHALL“, these are the issues resolved. With the exception of the first one, everything else is the same as the February patch:

  • Using JRockit with Weblogic Server 10.3.4 collaboration project access throws a TreeUnmrashall exception (12423316).
  • WebEdit does not work on IE8. (9723488)
  • Error in task submenu in IE8 (9712774)
  • Cannot upload multiple files in Collab (9274372)
  • Bulk upload JRE updated to 1.6 in order to support IE8 (8760280)
  • Notifications should be sent from a fixed address for creation and deletion (9647426) This fix involves updates to both collab and notification components.
  • Subscription to project overview causes conflicts with Immedate Subscriptions (8759642)
  • Non gatewayed webedit does not work (9434805)
  • File leak in Search Service during search index rebuild (11726008)
  • Webcenter Collaboration notifications to large numbers of recipients fail (9484113)

So, sure, this patch from May of last year supersedes that one from February. What’s strange, though, is that the link I provided in the post last year no longer works. Shouldn’t it have a message that says “this is no longer the latest patch version”, or “click here to see a list of all critical fixes released for this product”? In fact, where IS the definitely list of all of Oracle’s critical fixes? Seriously, does ANYONE know?

Don’t get me started on Oracle’s BS monthly email about all their “critical patches”. In fact, maybe you should, ’cause I’m this close to writing a Wall of Shame Rant about that ridiculousness. I mean, really, WHO is possibly helped by this?

Either way, stay tuned here for some reviews of the new Collab 10gR4 version, which presumably contains these patches (although, not totally guaranteed given the strange release history).

100% CPU in Collaboration Server on Solaris?

Tuesday, May 10th, 2011

Today’s post comes from a Rock Star Consultant in the WebCenter Consulting space.  It has to do with WCI Collaboration Server consuming 100% CPU on Solaris, but might be relevant to those Windows users out there.  While I personally haven’t experienced this particular issue at client sites (virtually all of them Windows), it sounds like if you’re running Collab in Unix, it might be worth upgrading your JVM.


Collab periodically starts chewing up CPU until it maxes out the box and ultimately dies.


This looks to ultimately be a problem at the JDK level.  Out of the box, the Tomcat Collab is deployed to uses JDK 1.5.  There’s a bug in JDK 1.5 that causes the NIO connector in tomcat to sometimes freak out, resulting in Collab spinning out of control and eating all the server CPU.  For details, see this thread:

Here’s the rundown on the diagnosis we did (Collab on Solaris)

Collab is eating up a huge amount of CPU minimal user load(80%+ on a server where it usually uses ~10%).

Troubleshooting performed:
1) Generated a stack trace for Collab
2) Run prstat -Lp <Collab Pid>.  This shows us how much CPU each thread in Collab is taking. Note that the top three threads in the attachment are taking up 22% of the CPU each.  Also note that those threads have used a huge amount of CPU time: 3-4 hours each).

3) Note the LWPID of each of the busy threads:  6248, 3413, 8198.
4) Now convert those numbers to Hex:
6248 -> 1868
3413 -> d55
8198 -> 2006
5) Now look for those hex thread ids in the thread dump.  You’ll see they all have the same stack (for example: nid=0xd55).  Specifically:


“http-7127-exec-23” daemon prio=10 tid=0x00000001008ee360
nid=0xd55 runnable [0xfffffffeff0fa000..0xfffffffeff0ff728]
at org.apache.coyote.http11. InternalNioOutputBuffer.addToBB(
– waiting to lock <0xffffffff40927c48> (a org.apache.coyote.http11.InternalNioOutputBuffer)
at org.apache.coyote.http11. InternalNioOutputBuffer.access$000(
at org.apache.coyote.http11. InternalNioOutputBuffer$SocketOutputBuffer.
at org.apache.coyote.http11. filters.ChunkedOutputFilter.doWrite(

Again, note that all the stack traces are the same and they appear to be trying to flush/close and output stream, but it looks like they’re blocked.

Update the Collab JVM to a recent release of the 1.6 JDK (1.6u23 for example).  Restart Collab

Prior to change, Collab was crashing multiple times a day and using at least 20% of the CPU on a beefy Solaris box.  Post change, no Collab crashes, and Collab is pretty consistently using about 4% of CPU on the server.

A fix for WCI Collaboration WebEdit using Office 2010

Tuesday, March 15th, 2011

As we saw a couple days ago, WebCenter Collaboration Server doesn’t work with Office 2010. The problem is rooted in the fact that WebDAV calls to Collab from Office 2010 aren’t strictly RFC-compliant because they’re missing a Content-Length header, and Tomcat rejects them because of that – even before Collab gets the HTTP requests.

Which got me to thinking: what if we could create an intermediate server that would add this header for us?   Then Tomcat would accept the connection and we’d be able to tell if Collab was able to “do its thing”. I started down the path of creating an Apache ProxyPass to do just this, when a friend and fellow Hak’er (Brian Hak) – who has been fighting his own fires with Collab these days – suggested that I try AJP. Even better, he pointed out, Collab’s default configuration settings allow you to turn this on by just flipping a switch; with it, we could then use mod_headers to add the Content-Length header.

Tip o’ the the hat to Brian for the AJP idea – it turns out that if Apache and AJP are handling the HTTP requests rather than Tomcat and its embedded web server, we don’t even need to muck with HTTP headers, as the request does NOT get blocked:

… and WebEdit (mostly) works in Office 2010!

I should warn you before proceeding: While I can absolutely say that WebEdit works in my test environment with Office 2010, .NET WCI Portal 10.3.0, on IIS 6, your mileage may vary, and I can’t guarantee this will work. I’ve found that Office 2007 has its own quirks – it works with the Collab Office Tools plugin in the ribbon bar, but not the out-of-the-box “save” implementation using Office’s own WebDAV calls. Perhaps that’s a post for another day, but early results show that while this hack doesn’t hurt Office 2007 support, it doesn’t seem to help either. Obviously, do extensive testing before even considering moving to a production environment.  Stay tuned for the next update in these pages, as there’s yet another database hack needed to make it fully functional.

Disclaimers out of the way, let’s dig in. Basically, what we’re going to do is:

  1. Install Apache Web Server on the Collab Machine, with the AJP plugin
  2. Configure Collab to accept AJP calls
  3. Configure the portal to send requests to Apache instead of directly to Collab (Tomcat)