Try this query to get hit counts for each card in the KD, and marvel in the awesomeness that is your popular documents – regardless of how many times your Analytics collector dies:

SELECT     ptc.name, ptcs.CARDID, ptcs.POPULARITY, ptcs.HITCOUNT, ptcs.LASTHIT
FROM         PTCARDSTATISTICS ptcs, PTCARDS ptc
WHERE     (LASTHIT > GETDATE() - 30) AND (HITCOUNT > 0) and (ptcs.cardid = ptc.objectid)
ORDER BY HITCOUNT DESC

Interesting side note – notice how all those document hit counts are a multiple of 10? The reason for this is a scalability approach that Plumtree came up with years ago. The idea is that rather than writing to the database every single time a document is downloaded, the code only increases the hit count 10% of the time, but increases the count by 10 each time. So, statistically, the counts are accurate over time, but the database is written to much less frequently.

You can view the original post for the source code and a description of how this filter works, but recently I’ve discovered that the MIME types in the original post should be updated for recent Office documents (.docx and .xlsx). Instead of “application/msword”, the config file should use the MIME types provided by Microsoft. Specifically, the configuration file for OpenerFilter should look more like the following:

<?xml version="1.0"?>
<IntegrystOpener>
 <default>
  <removeContentDisposition value="0" />
  <replaceContentDispositionWithInline value="1" />
  <fixContentType value="1" />
 </default>
 <extension-map>

  <doc value="application/msword" />
  <dot value="application/msword" />
  <dotx value="application/vnd.openxmlformats-officedocument.wordprocessingml.document" />
  <docx value="application/vnd.openxmlformats-officedocument.wordprocessingml.document" />
  <xlt value="application/vnd.ms-excel" />
  <xls value="application/vnd.ms-excel" />
  <xlst value="application/vnd.openxmlformats-officedocument.spreadsheetml.sheet" />
  <xlsx value="application/vnd.openxmlformats-officedocument.spreadsheetml.sheet" />

  <pdf value="application/pdf" />
  <gif value="application/octet-stream" />
  <jpg value="application/octet-stream" />
  <png value="application/octet-stream" />

 </extension-map>
</IntegrystOpener>

In our last post in this painful series, I shared some code that will resolve the ALUI / WCI Knowledge Directory links so that users can right-click and copy the links, and the files open as expected when clicked.

Since then, I’ve found that when an admin is in EDIT mode in the Knowledge Directory, the link calamity continues, and a completely different and incomprehensible linking mechanism is used. So for your reading pleasure today, I’ll update that post to handle edit mode as well as browse mode. Again, just add the following HTML to the footer used in your Knowledge Directory (depending on Experience Definition, there may be more than one):

<META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=UTF-8">
<pcs:value expr="text"></pcs:value>

<pt:common.transformer pt:fixurl="off" xmlns:pt='http://www.plumtree.com/xmlschemas/ptui/'/>
<script>
try
{
       // KD BROWSE Mode
       // <a href="#" onclick="var currentWin = PTCommonOpener.openInNewWindow ('', 'Opener_18_1214986', 800, 600, true); currentWin.location= 'http://portal.integryst.com/portal/server.pt /document/1214936/DocName'; return false;">DocName</a>
       if (document.title=="Directory")
       {
              // get all links
              var anchors = document.links;
              for (x=0; x<anchors.length; x++) {

                     // get only those with an onclick event
                     onclickJS = anchors[x].onclick;
                     if (onclickJS != null)
                     {
                           onclickJS = "" + onclickJS;
                           var locCommonOpener = onclickJS.indexOf("PTCommonOpener.openInNewWindow");
                           var locpos = onclickJS.indexOf("currentWin.location");

                           // only process links that have the broken PTCommonOpener.openInNewWindow
                           if ((locpos>=0) &amp;amp;&amp;amp; (locCommonOpener>=0))
                           {
                                  // extract the URL from the broken onclick function
                                  var url = onclickJS.substr(locpos);
                                  var locstart = url.indexOf("'");
                                  if (locstart>=0)
                                  {
                                         url = url.substr(locstart+1);
                                         var locend = url.indexOf("'");
                                         if (locend>=0)
                                         {
                                                // update the URL by clearing the onclick and setting the URL to open in a new window
                                                url = url.substring(0,locend);
                                                anchors[x].href=url;
                                                anchors[x].target="_new";
                                                anchors[x].onclick=null;
                                         }
                                  }
                           }
                     }
              }
       }

       // KD Edit Mode
       // <a href="#" onclick="var currentWin = PTCommonOpener.openInNewWindow ('', 'Opener_18_1214986', 800, 600, true); currentWin.location = PTCommonOpener.getOpenerURLOpenObjID (18, 1214926, '', 2); return false;"> DocName </a>
       if (document.title=="Edit Directory")
       {
              // get all links
              var anchors = document.links;
              for (x=0; x<anchors.length; x++) {

                     // get only those with an onclick event
                     onclickJS = anchors[x].onclick;
                     if (onclickJS != null)
                     {
                           onclickJS = "" + onclickJS;
                           var locCommonOpener = onclickJS.indexOf("PTCommonOpener.openInNewWindow");
                           var locpos = onclickJS.indexOf("currentWin.location");

                           // only process links that have the broken PTCommonOpener.openInNewWindow
                           if ((locpos>=0) &amp;amp;&amp;amp; (locCommonOpener>=0))
                           {
                                  // extract the URL from the broken onclick function
                                  var url = onclickJS.substr(locpos);
                                  var locstart = url.indexOf("=");
                                  if (locstart>=0)
                                  {
                                         url = url.substr(locstart+1);
                                         var locend = url.indexOf(";");
                                         if (locend>=0)
                                         {
                                                // update the URL by clearing the onclick and setting the URL to open in a new window
                                                url = url.substring(0,locend+1);
                                                anchors[x].href=eval(url);
                                                anchors[x].target="_new";
                                                anchors[x].onclick=null;
                                         }
                                  }
                           }
                     }
              }
       }
}
catch (err)
{
       // ignore errors
}
</script>

On a completely unrelated note, thanks to my wonderful numbering scheme of these posts, I can now count to four in French…

Configuration is identical to creating a Web Crawler. In administration, select “Create Object: Content Crawler – WWW” and choose the “World Wide Web” Content Source:

Here, instead of entering a web site, just provide the URL of the RSS feed:

Once the job runs, a card is created for each article in the feed:

Note the created date shows when the feed was crawled, not when the original articles were written. And in this example, only 11 cards have been created because that’s all that’s being provided on the Integryst RSS Feed. Both of these problems can be resolved by running your crawler job regularly, so that the dates are closer to when the posts are written, and the cards stick around after they’ve “left the feed”.

You got the “To help protect your security, Internet Explorer blocked this site from downloading files to your computer” blues, huh?

I’ve tried creating a Plumtree Filter, and that worked pretty well, but not quite enough.

I’ve tried tweaking the Portal’s Javascript files, and THAT worked pretty well, but again, not quite well enough.

So, today, my friend’s, third time’s a charm:  rather than trying to fix this on the server side, we’re going to knock out this issue once and for all on the client side.  Check out those previous blog entries for a more detailed description of the problem, but basically, it’s because the portal uses a crazy convoluted way of opening documents via javascript. 

What we’re going to do today is stick some javascript in the footer of the page.  This JavaScript is going to simply find all those back-asswards links and replace them with NORMAL <a href=xxx target=_new> links.  If you add it to the footer of your site (specifically, the footer used for the KD, but the code is smart enough not to do anything if it’s on all pages), it should be able to take care of the rest.  Just make sure you add it to the Presentation Template and not the Content Item, because you-know-who only knows what a mess the out-of-the-box rich text editor is with javascript and adaptive tags.

The code after the break should be self-explanatory, but as always, if you’ve got a question or comment, feel free to post it here.  Also note that this code only fixes the download links; it doesn’t kill that “Open/Save” dialog box for things like HTML files.  For that you’ll still need the Plumtree Filter.

<pt:common.transformer pt:fixurl="off" xmlns:pt='http://www.plumtree.com/xmlschemas/ptui/'/>
<script>
try
{
       // only run in the KD
       if (document.title=="Directory")
       {
              // get all links
              var anchors = document.links;
              for (x=0; x<anchors.length; x++) {

                     // get only those with an onclick event
                     onclickJS = anchors[x].onclick;
                     if (onclickJS != null)
                     {
                           onclickJS = "" + onclickJS;
                           var locCommonOpener = onclickJS.indexOf("PTCommonOpener.openInNewWindow");
                           var locpos = onclickJS.indexOf("currentWin.location");

                           // only process links that have the broken PTCommonOpener.openInNewWindow
                           if ((locpos>=0) &amp; &amp; (locCommonOpener>=0))
                           {
                                  // extract the URL from the broken onclick function
                                  var url = onclickJS.substr(locpos);
                                  var locstart = url.indexOf("'");
                                  if (locstart>=0)
                                  {
                                         url = url.substr(locstart+1);
                                         var locend = url.indexOf("'");
                                         if (locend>=0)
                                         {
                                                // update the URL by clearing the onclick and setting the URL to open in a new window
                                                url = url.substring(0,locend);
                                                anchors[x].href=url;
                                                anchors[x].target="_new";
                                                anchors[x].onclick=null;
                                         }
                                  }
                           }
                     }
              }
       }
}
catch (err)
{
       // ignore errors
}
</script>

Ever gotten this little present after doing an upgrade to 10gR3, when trying to open a document in the Knowledge Directory using IE7 or IE8?

Ah, the old “To help protect your security, Internet Explorer blocked this site from downloading files to your computer.”  What’s even more ridiculous, if you “accept” the download, it STILL doesn’t work because IE tries to reload the page, and WCI is very stupid about how it opens this popup window (see how that address bar shows “about:blank”?).  It seems this only happens in IE, and only if you have adaptive layouts disabled with “open documents in new window” enabled, so it doesn’t affect everyone.  But I die a little on the inside every time I see that stupid thing: yes, my friends, I am an empty, hollow shell of a consultant.

Fortunately, I decided to seek redemption for WCI and myself by proxy, and checked out the HTML source for a typical file open link.  It’s kinda hilarious:

<a href="#" onclick="var currentWin = PTCommonOpener.openInNewWindow('', 'Opener_18_1322446', 800, 600, true); currentWin.location= 'http://server/portal/server.pt/document/1322446/slide_1'; return false;">Doc Name</a>

Whhaaaaaaa…!?  Sure I know there are dozens of ways you can open a document in a popup window, and many have their advantages (for example, by using JavaScript you can control the size and layout of the window), but I’d be hard-pressed to come up with a worse way to open a document.  I’ve always hated the fact that you can’t just hover over the link to see the URL, and can’t right-click and copy the shortcut to mail to someone later. (Tip: use adaptive layouts and these problems go away!).  Or they could have at least just done a window.open method in that onclick event.  OK, so they wanted to use a wrapper method – that’s fine – but the PTCommonOpener.openInNewWindow actually takes a URL paramater, so they could have just passed the URL into that function, rather than creating a new, EMPTY document and then trying to use Javascript to load the document into that window.  At very least they could have put the document URL in the “href” parameter so it’d work without JavaScript (and be 508-compliant) and users could copy the links with a simple right-click.

OK enough rambling.  On to a solution:

It’s clear looking at the above code that while Microsoft’s security controls can sometimes be oppressive, this blocked transfer kinda makes sense:  IE blocks downloads that weren’t initiated by the user (i.e., were initiated by JavaScript).  The problem is, given the brain-dead way that documents are opened in a new window (by creating an EMPTY window first, then trying to download the document later), when you accept the download, the browser reloads the about:blank() page instead of the actual file.

Using a .NET decompiler (I used dis# because I needed to search the entire portaluiinfrastructure.dll for the text string “openInNewWindow”), I found the code in com.plumtree.portaluiinfrastructure.opener.PTOpenerLinks:

public static string GetJSOpenInNewWindow(string _strURL, string _strWindowName, string _strWindowWidth, string _strWindowHeight, bool _bIsWindowFancy)
        {
            com.plumtree.openfoundation.util.XPStringBuilder sbOpenerJS = new com.plumtree.openfoundation.util.XPStringBuilder(300);
            sbOpenerJS.Append("var currentWin = ");
            sbOpenerJS.Append("PTCommonOpener.openInNewWindow");
            sbOpenerJS.Append("('', '");
            sbOpenerJS.Append(_strWindowName).Append("', ");
            sbOpenerJS.Append().Append(", ");
            sbOpenerJS.Append(_strWindowHeight).Append(", ");
            sbOpenerJS.Append(_bIsWindowFancy);_strWindowWidth
            sbOpenerJS.Append(");");
            sbOpenerJS.Append("currentWin.location=").Append(_strURL);
            return sbOpenerJS.ToString();
        }

Ideally, I’d change the code to something like this:

public static string GetJSOpenInNewWindow(string _strURL, string _strWindowName, string _strWindowWidth, string _strWindowHeight, bool _bIsWindowFancy)
        {
            com.plumtree.openfoundation.util.XPStringBuilder sbOpenerJS = new com.plumtree.openfoundation.util.XPStringBuilder(300);
            sbOpenerJS.Append("var currentWin = ");
            sbOpenerJS.Append("PTCommonOpener.openInNewWindow");
            sbOpenerJS.Append("('");
            sbOpenerJS.Append(_strURL);
            sbOpenerJS.Append("', '");
            sbOpenerJS.Append(_strWindowName).Append("', ");
            sbOpenerJS.Append().Append(", ");
            sbOpenerJS.Append(_strWindowHeight).Append(", ");
            sbOpenerJS.Append(_bIsWindowFancy);_strWindowWidth
            sbOpenerJS.Append(");");
            return sbOpenerJS.ToString();
        }

(or even better, figure out how to change this to a normal link like <a href=”http://url” target=_new>docname</a>)

… but recompiling decompiled code – especially in a .NET library as large as portaluiinfrastructure – well, that just wasn’t going to happen.

So we can’t change the HTML that generates this terrible link, but is there at least a way to fix the problem where we can get the browser to open the link in a new window in one step instead of two?  Fortunately, the answer is yes.  I’ll leave the analysis of this hack to you, but basically it creates a global object and allows both of those calls (open the window and set the URL) to be made before actually calling the open window method.

Open the PTHOME\imageserver\plumtree\portal\private\js\ptcommonopener.js file and change:

PTCommonOpener.openInNewWindow = function(_strURL, _strWindowName, _strWidth, _strHeight, _bIsFancyWindow)
{
 return PTCommonOpener.JSUTIL_VERSIONOBJ.PTWindowUtil.openWindow(_strURL, _strWindowName, _strHeight, _strWidth, _bIsFancyWindow);
}

… to:

var newwin = {location:'', _strWindowName:'', _strWidth:'', _strHeight:'', _bIsFancyWindow:''};
PTCommonOpener.handlePopup = function() {
 PTCommonOpener.JSUTIL_VERSIONOBJ.PTWindowUtil.openWindow(newwin.location, newwin._strWindowName, newwin._strHeight, newwin._strWidth, newwin._bIsFancyWindow);
}

PTCommonOpener.openInNewWindow = function(_strURL, _strWindowName, _strWidth, _strHeight, _bIsFancyWindow)
{
 if (_strURL != "")
  return PTCommonOpener.JSUTIL_VERSIONOBJ.PTWindowUtil.openWindow(_strURL, _strWindowName, _strHeight, _strWidth, _bIsFancyWindow);

 newwin.location=_strURL;
 newwin._strWindowName=_strWindowName;
 newwin._strWidth=_strWidth;
 newwin._strHeight=_strHeight;
 newwin._bIsFancyWindow=_bIsFancyWindow;
 window.setTimeout("PTCommonOpener.handlePopup();",50);
 return newwin;
}

Update 7/17/2010: Thanks to Joel Collins for helping out with some of the code here and building the solution’s binaries! Hopefully you find them useful so you can just deploy the .DLL without having to compile it yourself. Download DisplayFilter.zip for the binaries (source and config files included).

There are two main problems: poorly implemented document open handlers (which I’ll discuss in my next post), and that pesky Content-Disposition header that tries to force the browser to throw up an “Open/Save” dialog instead of just opening inline (which is definitely useful for HTML and Office files in particular, where users just want to view the documents in their browser without having to save ‘em first).  A third but ancillary issue is that occasionally the portal gets the MIME type of the document wrong in the Content-Type header, which is typically using application/octet-stream and doesn’t give the browser any clue about what type of file it is.

In the past posts I’ve described using BigIP iRules to remove the Content-Disposition header, but I ran into another client issue recently where just killing this header isn’t enough.  Take the following response headers for a document transfer that is NOT using the iRule to kill the Content-Disposition header:

This PowerPoint file, when clicked on in the portal, will throw the open/save dialog, but what if we wanted it to give the browser the option to open in-line?  We’d get rid of that Content-Disposition header.  Here’s the problem:  if we do that, the portal is returning a Content-Type of application/octet-stream.  Which means, without the file extension in Content-Disposition, the browser has no idea what type of document this is, so it throws up a “FIND PROGRAM/Save” dialog box instead of an “OPEN/Save” dialog box.

The solution to this problem would be to drop the Content-Disposition header, but use the file extension to fix the Content-Type while we’re at it (to application/vnd.ms-powerpoint).  But now our iRules are getting complicated, and not everyone has BigIP anyway.  Fortunately, my love/hate relationship with support.oracle.com shifted much more to the former when i found this: How to Eliminate the ‘Save/Open with’ Prompt when Opening HTML Files [ID 971003.1].  Then, of course, it shifted back to the latter when I realized that I found this article by complete dumb luck navigating the terrible interface, and the article is nowhere to be found searching through Google. But that’s another story.

At any rate, a solution was clear: you can write a .NET or Java FILTER to manipulate headers before they’re returned to the browser!  As such, I took the code from that article and expanded on it with a .NET filter of my own, which will:

1. Remove the Content-Disposition header selectively on different file types
2. Change the Content-Disposition header to use “inline” instead of “attachment” instead of just deleting the header (this seems to work with some browsers; a brief discussion on the differences is here)
3. Fix the Content-Type header if the file name extension doesn’t map the MIME type the portal thinks it is
4. Do all this through a filter and a dynamically loadable varpack that allows different configurations based on the type of file being served.

So while it’s a bit optimized in the sense that it only looks at responses being gatewayed through the portal, it’s still kind of a hack because personally, I think this should be an option in the portal in the first place rather than adding the additional overhead of a filter.

But, nonetheless, it seems to work well, and you can download the code here and the varPack file here.

If you’d like assistance with actually building and deploying this, or porting to Java – drop me a line.

Instead, I’ll give you a quick tip to answer what should be a relatively simple question: How do I replace an existing document that has been uploaded to the WCI Knowledge Directory? At first glance, it’d seem you should just upload a new document and delete the old one. The problem is, especially if your site is large or exposed publicly on the Internet, someone may have linked to the original URL. This URL contains the Object ID of the card in there, so the new file – even if it has the same name – will have a different link. This is even true, for the most part, with Friendly URLs that were introduced in ALUI 6.5 and expanded in WCI 10gR3.

The bad news is that there is no way through the UI to change the file itself; the good news it’s an easy file swap that can be done on the server containing the Document Repository. Just open the properties page for the file, and look at the “Document Upload DocID” field (and the “Document Upload Repository Server”). Then, take your new document, rename it exactly as shown in the properties, and put replace the file in that directory. In the example below, this file would need to be called “D004783E.ACT” in the DR folder at “documents/ptupload/active/F0080AF6/”:

On the other hand, this advice can bite you.  Have you ever gotten an “Error displaying Dropdown menu tabs” error in the nav bar when using the JS drop-down menus?

Well, in at least one case the problem is simple:  Give that EVERYONE (or at least the respective GUEST users) access to the root directory of your Knowledge Directory.  No need to give that user access to any cards or folders – just the root directory itself.  It seems the code in 10gR3 chokes if it can’t read this directory when trying to display the Directory drop-downs.  In fact, this is similar to Bill Benac’s Bug about missing footers in 10gR3 breaking the page (incidentally, I’ve run into the same problem with headers) – it seems 10gR3 isn’t quite as “forgiving” as previous versions were regarding error handling.

This quick fix should restore your menu:

… but there are other reasons you can run into this error.  I’ve got another post pending on some other root causes of this problem (spoiler alert: give SELECT privileges to the EVERYONE group for your headers!); in it I’ll offer some instructional advice on how to diagnose the root cause of problems like this.

In this case, after downloading the Excel file the first time, I confirmed it was in my Temporary Internet Files folder:

I was also able to verify that the portal was returning a “Not-Modified” header when the user tried to open this Excel document again:

The fix I applied here?  Another BigIP iRule that turns off caching of Excel files – after the jump.

The BigIP iRule is:

if { [HTTP::header Content-Type] contains "ms-excel"} {
    HTTP::header replace "Expires" "-1"
    HTTP::header remove "ETag"
 }

… and as you can see, the HTTP headers now make sure the content isn’t cached, which prevents Excel from failing when opening cached documents:

This fix doesn’t feel like the best approach, but it has worked for me.  Have you had this problem yourself?  Got a better resolution/workaround?  Let us know in the comments!