The first one you will be able to view.  It is a pastebin clone hosted on google app engine (my first dive into python, besides this facebook app that i made – also hosted on google app engine).  I chose GAE because it’s a relatively new entrant into the whole “cloud” hosting game, and because its insanely scalable (although i doubt i’d need to worry about that).  The pastebin supports highlighting in tonnes of languages (way more than paste2.org), and uses FBJS and jQuery stuff for its frontend ajax.  It also supports submission via command line applications that users create (because it returns its output in very standard JSON).  If you want to check it out, it’s located at http://pastebox.appspot.com/.  Oh, and for its backend, it uses the django templating system (very, very handy – it’s a shame that the django Model’s system doesnt work on GAE otherwise you’d be able to use the admin interface).

Second item i’ve been working on is a facebook application, as mentioned above.  It’s not public yet and i dont ever expect it to be because it was a practice in python, and it doesnt really utilise django’s template system as much as it could have.

Third item, and the most recent (and fun) is a PHP RRDtool grapher.  You might think, “yes, but there are plenty of those out there already, nothing new here.”, but you would be wrong.  This actually renders the graphs on the CLIENTS side, instead of getting RRDtool to generate an image every 10 minutes or whatever.  This came about because the only linux box i’ve got sitting here is a WD Mybook World Edition II, and as anyone would tell you they have fairly limited specifications (32mb of ram, 64 if you’re game to screw around with the bootloader), and a really, really crappy cpu.  Because of this, i used the Mybook in conjunction with MRTG/RRDTool to generate images of the traffic going through my dlink (*cough*) router.

There were obvious performance concerns from the get-go.  To alleviate the strain on the mybook, when i didn’t look at the graphs, i setup a ramdisk on it so the hard drives wouldnt have to spin up, and i also got a script that generated images whenever the user requested them (so there were no images pre-stored).  I tried a PHP version, and a Perl version with little success, both being as bad as each other (probably took around 5 seconds to render a page with 5 graphs on it).

To alleviate this, i thought it would be easier to render the images on the client side and just pass the client some javascript/json datapoints in an array which would tell the browser what to graph.  The first concept of this worked excellently.  I simply ran “rrdtool fetch blah.rrd….” whenever a user loaded the page, and then sent that in an array to the web browser.  To graph, i used the new “canvas” feature of HTML5 (more specifically, the flot graphing library).  Those of you that are in-the-know, might know what challenges HTML5 presents – primarily, lack of compatability.  The web-browsers i tested the graphing app in were: Firefox 3.5, and Chrome 2.0, however, a friend using the latest safari (i assume) reported that it worked fine on Mac OSX.  Pretty much any web browser that supports HTML5 can view the page, any that cant – tough luck.  This also has some advantages as well, because we know the browser is recent, we don’t have to worry about supporting older browsers like IE6/7 (don’t know if 8 chokes on the graphing app, havent tested with it).

If you want to skip the next paragraph, screenshots are at the bottom of the page.

The next features I added to it were the ability to switch between different periods (daily, hourly, monthly, etc etc).  That was relatively easy, and was accomplished in a few short hours using jQueryUI for the tabbing library.  There was another advantage to using jQueryUI, it handled fetching the graph page transparently via ajax which meant less work for me.  Next features i added were the ability to graph multiple RRD’s.  I should note that every new feature added didn’t decrease the performance of the application overall by much, compared to the regular static ones.

Once the code for graphing multiple RRD’s was done, i rewrote the way the graph data was sent to the browser so i could enable/disable certain datasets on the fly (note: all of this is supported out of the box with the ‘flot’ graphing library.  Truly the best i’ve seen).  It wasn’t long until i decided to jQueryUI everything (so you change one stylesheet, or make your own using the jQueryUI ThemeRoller and the looks of the tab ui change completely).

Anyway, i’m running out of ideas of what to add next.  One idea is to fetch data from the server every time a client zooms on a graph (at the moment, it only redraws that section – and due to performance constraints, i can’t send every datapoint to the user initially, otherwise for daily graphs you’d have 40 datapoints or so, depending on how often your MRTG cronjob runs).

If you are wondering the obvious: “When/where can I download this?”  the answer at the moment is: “You can’t”.  If you would like a copy, email me (brent.pickup [-] gmail [.] com) and I’ll usually get back to you within 24hrs.  I’ll give it out on a case-by-case basis depending on your use for it (if it’s commercial a fee might be attached).

By the way, if you’re curious: yes, this can be adapted for things other than data going in/out of a router.  These could be memusage, cpu usage, etc etc.  However, i haven’t gotten around to testing that yet because i haven’t got a cpu usage script for MRTG.

View video in HD at youtube.com if you can’t see it (right click on the video –> view at youtube).

A description of the screenshots below (left, right then down):

1. A regular RRD graph containing two RRD files.
2. Demonstrating the jQueryUI dropdown menu.
3. RRD Selector – appears on the index page, and on the left of every graph page.
4. Closeup of selector with two options ticked.
5. Another regular RRD Graph
6. Two datasets unchecked.
7. Demonstrating the “zoom” feature.


--- fs_storage.py 2008-06-29 22:22:26 +1000
+++ fs_storage_ps3.py 2008-07-13 14:33:43 +1000
@@ -16,6 +16,7 @@
import mimetypes
mimetypes.init()
mimetypes.add_type('video/mp4', '.mp4')
+mimetypes.add_type('video/divx', '.avi')

@@ -556,6 +557,8 @@
'http-get:*:video/x-msvideo:*',
'internal:%s:video/avi:*' % self.server.coherence.hostname,
'http-get:*:video/avi:*',
+ 'internal:%s:video/divx:*' % self.server.coherence.hostname,
+ 'http-get:*:video/divx:*',
'internal:%s:video/quicktime:*' % self.server.coherence.hostname,
'http-get:*:video/quicktime:*'],
default=True)


Patched Coherence

Enjoy.

SPDIF - AC3 Encoder Bitrate: 320kbps, Use AC3 Encoder.

SPDIF Mode: SPDIF Wrapped.

SPDIF Passthrough: (untick all 3).  This allows you to reroute seperate channels on a 2.0/2.1 channel broadcast, and then upmix it to 5.1 (this is what i do).

—

That configuration was for standard definition channels, which allows upmixing and re-encodes audio.  I prefer that one because it sounds ‘crisper’ than using the raw mpeg audio.

—

This configuration is for true 5.1 programmes:

SPDIF - Disable AC3 Encoder.

SPDIF Mode: SPDIF Wrapped.

SPDIF Passthrough: AC3 Ticked.

—

That option will pass the untouched 5.1 channel audio from the tv station directly through to your speakers/decoder.  Using these configurations, you can easily switch between upmixed and original audio.

Before i finish, i just have to say that it taken me days on google to figure this problem out, so i hope it helps someone else in the future.

   1: #include "stdafx.h"

   2: #include <windows.h>

   3: #include <httpfilt.h> 

   4: #include <shlwapi.h>

   5: #include <atlstr.h>

   6: #pragma comment(lib, "shlwapi.lib")

   7:  

   8: #define DEFAULT_BUFFER_SIZE 2048

   9: #define MAX_BUF 2048

  10: char szUsername[DEFAULT_BUFFER_SIZE];

  11: char *pszUsername = szUsername;

  12: const char szIndexFiles[8][13] = {"index.html","index.asp","index.aspx","default.htm","default.html","index.php","default.php","index.mspx"};

  13: char pszDebugEnabled[1024];

  14: char pszHomePath[1024];

  15: DWORD g_dwAppendBufferLen = 0;

  16:  

  17: bool DoInitialize(HMODULE hModule);

  18: DWORD DoUrlRewriting(HTTP_FILTER_CONTEXT *pfc, HTTP_FILTER_URL_MAP *pMap);

  19:  

  20: #ifdef _MANAGED

  21: #pragma managed(push, off)

  22: #endif

  23:  

  24: #define DLC_TRACE(x) DlcReportEventA(EVENTLOG_INFORMATION_TYPE, x)

  25:  

  26: void DlcReportEventA(WORD wType, LPCSTR pszMessage)

  27: {

  28:     if (pszDebugEnabled == NULL || pszDebugEnabled != "True") {

  29:         return;

  30:     } // we don’t write anything.

  31:  

  32:     HANDLE  hEventSource;

  33:     LPCTSTR  lpszStrings[1];

  34:  

  35:     lpszStrings[0] = (LPCTSTR)pszMessage;

  36:  

  37:     /* Get a handle to use with ReportEvent(). */

  38:     hEventSource = RegisterEventSourceA(NULL, "IISHome");

  39:     if (hEventSource != NULL)

  40:     {

  41:         /* Write to event log. */

  42:         ReportEventA(hEventSource, wType, 0, 0, NULL, 1, 0, (LPCSTR*) &lpszStrings[0], NULL);

  43:         DeregisterEventSource(hEventSource);

  44:     }

  45: }

  46:  

  47: BOOL GetRegistryKey (char *lpszConfigItem, char *outBuffer)

  48: {

  49:     char szRegistryResult[1024];

  50:     HKEY hKey;

  51:     BOOL bFoundItem = TRUE;

  52:     DWORD baseLen = 1024;

  53:     memset(szRegistryResult, 0, 1024);

  54:     char dbgInfo[DEFAULT_BUFFER_SIZE];

  55:     sprintf(dbgInfo, "DEBUG: Value Name: %s", lpszConfigItem);

  56:     DLC_TRACE(dbgInfo);

  57:  

  58:     if (RegOpenKeyEx(HKEY_LOCAL_MACHINE, "SOFTWARE\\BrentP\\IISHome", 0, KEY_READ, &hKey) != ERROR_SUCCESS) {

  59:         bFoundItem = FALSE;

  60:         DLC_TRACE("DEBUG: RegOpenKeyEx(…) failed to open key.");

  61:         return bFoundItem;

  62:     }

  63:  

  64:     if (bFoundItem == TRUE) {

  65:         /* Get Registry Value */

  66:         DLC_TRACE("DEBUG: Key found - ready for RegQueryValueEx(…)");

  67:         if (RegQueryValueEx(hKey, lpszConfigItem, NULL, NULL, (LPBYTE)szRegistryResult, &baseLen) != ERROR_SUCCESS) {

  68:             bFoundItem = FALSE;

  69:             DLC_TRACE("DEBUG: RegQueryValueEx(…) failed. No Value found.");

  70:         } else {

  71:             DLC_TRACE("DEBUG: upto strcpy(outBuffer, szRegistryResult)");

  72:             strcpy(outBuffer, szRegistryResult);

  73:         }

  74:     }

  75:  

  76:     RegCloseKey(hKey);

  77:     DLC_TRACE("DEBUG: RegCloseKey(hKey)");

  78:  

  79:     return bFoundItem;

  80: }

  81:  

  82: // End Registry Stuff

  83:  

  84: BOOL WINAPI __stdcall GetFilterVersion(HTTP_FILTER_VERSION *pVer)

  85: {

  86:     /* Specify the types and order of notification */

  87:  

  88:     pVer->dwFlags = (SF_NOTIFY_ORDER_HIGH | SF_NOTIFY_NONSECURE_PORT | 

  89:         SF_NOTIFY_SECURE_PORT | SF_NOTIFY_URL_MAP);

  90:  

  91:     pVer->dwFilterVersion = pVer->dwServerFilterVersion;

  92:     strcpy(pVer->lpszFilterDesc, (LPCSTR)"IISHome version 1.0");

  93:     DLC_TRACE("DEBUG: Started IISHome");

  94:  

  95:     // Now Init Registry Values

  96:     if (GetRegistryKey("HomePath", pszHomePath) == FALSE) {

  97:         strcpy(pszHomePath, "C:\\Users\\%s\\%s");

  98:     }

  99:     if (GetRegistryKey("DebugEnabled", pszDebugEnabled) == FALSE) {

 100:         strcpy(pszDebugEnabled, "False");

 101:     }

 102:  

 103:     char dbgInfo[DEFAULT_BUFFER_SIZE];

 104:     sprintf(dbgInfo, "DEBUG: HomePath: %s DebugEnabled: %s", pszHomePath, pszDebugEnabled);

 105:     DLC_TRACE(dbgInfo);

 106:  

 107:     return TRUE;

 108: }

 109:  

 110: BOOL FileExists(char *fileName) {

 111:     return GetFileAttributes(fileName) != INVALID_FILE_ATTRIBUTES;

 112: }

 113:  

 114: BOOL DirectoryExists(char *fileName) {

 115:     return GetFileAttributes(fileName) == FILE_ATTRIBUTE_DIRECTORY;

 116: }

 117:  

 118: DWORD DoUrlRewriting(HTTP_FILTER_CONTEXT *pfc, HTTP_FILTER_URL_MAP *pMap)

 119: {

 120:     char *s;

 121:     //unsigned int i; //allow conversion between size_t and int.

 122:     size_t i; // size_t is unsigned int. converting to int might cause loss of data.

 123:     DWORD dwRet = SF_STATUS_REQ_NEXT_NOTIFICATION;

 124:     /* Heres what we do:

 125:     1) Strip /~

 126:     2) Strip remaining /….. to get username alone.

 127:     3) Set Path to C:\Users\<username\

 128:     4) If any of the index files exist, strcat it on.

 129:     5) Now strcpy the physical path to the new path, and pass a return value.

 130:     */

 131:  

 132:     if (pfc == NULL || !pMap)

 133:     {

 134:         return dwRet;

 135:     }

 136:  

 137:     char szPhysicalPath[DEFAULT_BUFFER_SIZE];

 138:     DWORD cbBuf = DEFAULT_BUFFER_SIZE;

 139:  

 140:     if (!pszHomePath || pMap->pszURL[0] != ‘/’ || pMap->pszURL[1] != ‘~’) {

 141:         char dbgInfo[DEFAULT_BUFFER_SIZE];

 142:         sprintf(dbgInfo, "DEBUG: Could not match URL: %s", pMap->pszURL);

 143:         DLC_TRACE(dbgInfo);

 144:         return dwRet;

 145:     }

 146:  

 147:     // only match /~, not /bleh/~user.

 148:     char dbgInfo[DEFAULT_BUFFER_SIZE];

 149:     sprintf(dbgInfo, "DEBUG: URL Matched: %s", pMap->pszURL);

 150:     DLC_TRACE(dbgInfo);

 151:  

 152:     strcpy(szUsername, (pMap->pszURL)+2);

 153:     if (s = strchr(szUsername, ‘/’)) {

 154:         *s = (char)0; // replace it with a null character.

 155:         s++;

 156:         sprintf(szPhysicalPath, pszHomePath, szUsername, s);

 157:         char dbgInfo[DEFAULT_BUFFER_SIZE];

 158:         sprintf(dbgInfo, "DEBUG: Preliminary Physical Path: %s", szPhysicalPath);

 159:         DLC_TRACE(dbgInfo);

 160:     } else {

 161:         // we have nothing after the last /, so its http://server/~user/

 162:         sprintf(szPhysicalPath, pszHomePath, szUsername, "");

 163:         char dbgInfo[DEFAULT_BUFFER_SIZE];

 164:         sprintf(dbgInfo, "DEBUG: Preliminary Physical Path: %s", szPhysicalPath);

 165:         DLC_TRACE(dbgInfo);

 166:     }

 167:  

 168:     for (s = szPhysicalPath; *s; s++) {

 169:         if (*s == ‘/’) {

 170:             *s = ‘\\’;

 171:         } // replace / with a backslash

 172:     }

 173:  

 174:     DLC_TRACE("DEBUG: Replaced Slashes");

 175:  

 176:     // replace last character with a null value.

 177:     if (szPhysicalPath[i=strlen(szPhysicalPath)-1] == ‘\\‘) {

 178:         szPhysicalPath[i] = (char)0;

 179:         DLC_TRACE("DEBUG: Replaced Nulls");

 180:     }

 181:  

 182:     // FindFirstFile is inefficient, we use GetFileAttributes inside DirectoryExists.

 183:     if (szIndexFiles && DirectoryExists(szPhysicalPath)) {

 184:         DLC_TRACE("DEBUG: DirectoryExists(szPhysicalPath)=TRUE & szIndexFiles defined");

 185:         if (szPhysicalPath[i=strlen(szPhysicalPath)-1] != ‘\\’) {

 186:             strcat(szPhysicalPath, "\\"), i++;

 187:         }

 188:  

 189:         for (int j = 0; j < 9; j++) {

 190:             char szFName[2048];

 191:             strcpy(szFName, szPhysicalPath);

 192:             strcat(szFName, szIndexFiles[j]);

 193:  

 194:             if (FileExists(szFName)) {

 195:                 strcpy(szPhysicalPath, szFName);

 196:                 char dbgInfo[DEFAULT_BUFFER_SIZE];

 197:                 sprintf(dbgInfo, "DEBUG: Found: %s", szFName);

 198:                 DLC_TRACE(dbgInfo);

 199:                 break;

 200:             }

 201:         }

 202:     }

 203:  

 204:     if( strlen(szPhysicalPath) < pMap->cbPathBuff) {

 205:         strcpy(pMap->pszPhysicalPath, szPhysicalPath);

 206:         char dbgInfo[DEFAULT_BUFFER_SIZE];

 207:         sprintf(dbgInfo, "DEBUG: Final Path: %s", pMap->pszPhysicalPath);

 208:         DLC_TRACE(dbgInfo);

 209:     } else {

 210:         DlcReportEventA(EVENTLOG_INFORMATION_TYPE, (LPCSTR)"DEBUG: Couldnt remap URL - Not enough Memory");

 211:     }

 212:  

 213:     return dwRet;

 214: }

 215:  

 216: DWORD WINAPI __stdcall HttpFilterProc(HTTP_FILTER_CONTEXT *pfc, DWORD NotificationType, 

 217:                                       VOID *pvData)

 218: {

 219:     DWORD   status = SF_STATUS_REQ_NEXT_NOTIFICATION; 

 220:     switch (NotificationType)

 221:     {

 222:  

 223:     case SF_NOTIFY_URL_MAP:

 224:         status = DoUrlRewriting(pfc, (HTTP_FILTER_URL_MAP*)pvData);

 225:         break;

 226:     }

 227:     return status;

 228: }

 229:  

 230: bool DoInitialize(HMODULE hModule)

 231: {

 232:     return true;

 233: }

 234:  

 235: BOOL APIENTRY DllMain(HMODULE hModule, DWORD  ul_reason_for_call, LPVOID lpReserved)

 236: {

 237:     switch (ul_reason_for_call)

 238:     {

 239:     case DLL_PROCESS_ATTACH:

 240:         {

 241:             if (!DoInitialize(hModule))

 242:             {

 243:                 return FALSE;

 244:             }

 245:             break;

 246:         }

 247:     case DLL_PROCESS_DETACH:

 248:         {

 249:             g_dwAppendBufferLen = 0;

 250:         }

 251:         break;

 252:     }

 253:     return TRUE;

 254: }

 255: #ifdef _MANAGED

 256: #pragma managed(pop)

 257: #endif

Below is the registry file you’ll need to apply for it to work:

   1: REGEDIT4

   2:  

   3: [HKEY_LOCAL_MACHINE\SOFTWARE\BrentP\IISHome]

   4: "HomePath"="C:\\Users\\%s\\%s"

   5: "DebugEnabled"="False"

You’ll also need (because ISAPI requires some stuff to be exported) IISHome.def:

   1: LIBRARY    "IISHome"

   2:  

   3: ;DESCRIPTION  ‘IISHome Version 1.0′

   4:  

   5: EXPORTS

   6:     HttpFilterProc

   7:     GetFilterVersion

Once you’ve shoved all that into a visual studio project, you should be able to compile it — providing you’ve linked in Advapi32.lib for registry access and set the project to use a Multibyte character set, and only to use standard windows libraries (this can be done in the VS Project configuration dialogs).  Once you do this, the addon should compile with 0 warnings and errors, and then you can load it into IIS by right clicking on the website you want to load it in, then going to ISAPI Filters, then adding IISHome.  You will see any errors generated by IISHome in the application section of the error log (eventvwr.msc in Start -> Run).

You WILL also need to install the runtimes for whatever version of visual studio you compiled, for example, VS2005 or VS2008.  Enjoy.

This will probably be the last post for a while — again.

• Grid View (plus, no ugly text box underneath grid view anymore.)
• Now View (a new view which displays what tv shows are on now, or starting soon on all channels)
• Search View (so you can search for tv shows)
• and of course, our default listings view.
• Better support for EPGStream.net feeds.
• Cleaner display of data (in my opinion).
• Stores settings in an xml file, not the registry (so they are much more portable).

There is a downside with this version though.  You will not be able to select a default feed from the installer.  That was removed because the previous version used the deployment wizard in Visual Studio, whereas the new version uses the WiX Project from Sourceforge for the installer framework.  As i said before, some features are not back in the new version (like channel logos, but they will be soon).  Some of the menu option’s also haven’t been linked up to their respective sections, so if that occurs, give me a yell.

Enjoy.

You can get it at: http://brentp.net/downloads/projects/xmltv/Installer.msi

The old version will remain up at the old url until i get all of the old features into the new one (minus the text colouring features and so on).

public int NewIndexRecovery()
{
    // this creates an entirely new index.  Pretty much the following:
    /* <Data>
     * </Data>
     */
    System.IO.File.Delete(Properties.Settings.Default.IndexDocument);
    XmlDocument xNewIndex = new XmlDocument();
    XmlNode xData = xNewIndex.CreateNode(XmlNodeType.Element, “Data”, null);
    XmlNode xComment = xNewIndex.CreateComment(“Index File - Created: ” + DateTime.Now.ToString());
    XmlNode xCommentTwo = xNewIndex.CreateComment(“V” + strVersion + “DATAFILE”);
    xData.AppendChild(xComment);
    xData.AppendChild(xCommentTwo);
    xNewIndex.AppendChild(xData);
    xNewIndex.Save(Properties.Settings.Default.IndexDocument);
    return 0;
}

That code pretty much creates a document similar to the following:

The .NET framework handles all the pretty formatting as well, so you are actually able to read the files after you’re done with them.  Pretty cool huh?  Also, i have made an update to the XMLTV Viewer program, however, due to VS’s MSI maker thing complaining about missing bitmap files to build an installer, i am unable to upload it at the current time.  The new features in the current version are:

- EPGStream.net functionality (so you just enter in your epgstream id’s, and it’s ready to go.  The functionality can be turned on and off in the settings area, just remember that its a <LOT> buggy, so it may take some time to work.  Don’t try to download EPGStream feeds via the Settings -> XMLTV Location Downloader, because you will crash it, rather, use the main one in the main menu.  If you get a marquee progress bar (just continues going across, rather than an actual percentage), then its fetching an epgstream url.

- Bugfixes.

 EDIT: It’s just been uploaded at the usual url.  This will probably be the last build of it for some time (perhaps i might just recode the thing from scratch).

Download Link - XMLTV Viewer - LAST BUILD.

Same url as usual,

Enjoy.

Download Link - XMLTV Viewer

Improvements:

• We only parse the file once now, so this will hopefully go a long way in reducing memory and cpu usage.
• A Way to display extended information in grid view has been discovered, however, it is too taxing on the cpu. Might find a better method for the next version (using the Button/PictureBox/ButtonBase classes and overlaying that on the timeline)
• Bug fixed where the whole timeline would get f’ed up if you went to any other view and switched back to the timeline view.
• Removed old and unused code.
• Improved the way that the extended info is displayed in the textbox below the table.  Before it was statically coded, now it determines which stuff has no information (ie: some shows might have a "Ep title" tidbit, whereas some shows might not).

Improvements for the next version:

• I might actually read the xmltv spec <cough>
• Get the extended info working in grid view.

Same link as usual. enjoy.