So, what have you been upto?

So, as you can no doubt tell, this is the first update for a while. I’ve pretty much been sitting here coding stuff for a couple of months, so here are my most recent creations:

The first one you will be able to view.  It is a pastebin clone hosted on google app engine (my first dive into python, besides this facebook app that i made – also hosted on google app engine).  I chose GAE because it’s a relatively new entrant into the whole “cloud” hosting game, and because its insanely scalable (although i doubt i’d need to worry about that).  The pastebin supports highlighting in tonnes of languages (way more than paste2.org), and uses FBJS and jQuery stuff for its frontend ajax.  It also supports submission via command line applications that users create (because it returns its output in very standard JSON).  If you want to check it out, it’s located at http://pastebox.appspot.com/.  Oh, and for its backend, it uses the django templating system (very, very handy – it’s a shame that the django Model’s system doesnt work on GAE otherwise you’d be able to use the admin interface).

Second item i’ve been working on is a facebook application, as mentioned above.  It’s not public yet and i dont ever expect it to be because it was a practice in python, and it doesnt really utilise django’s template system as much as it could have.

Third item, and the most recent (and fun) is a PHP RRDtool grapher.  You might think, “yes, but there are plenty of those out there already, nothing new here.”, but you would be wrong.  This actually renders the graphs on the CLIENTS side, instead of getting RRDtool to generate an image every 10 minutes or whatever.  This came about because the only linux box i’ve got sitting here is a WD Mybook World Edition II, and as anyone would tell you they have fairly limited specifications (32mb of ram, 64 if you’re game to screw around with the bootloader), and a really, really crappy cpu.  Because of this, i used the Mybook in conjunction with MRTG/RRDTool to generate images of the traffic going through my dlink (*cough*) router.

Continue reading ‘So, what have you been upto?’

Coherence Patch

If you’ll see my post from earlier today, i’ve been playing around with coherence. Below is a PATCH for the 0.5.8 distribution to get DivX working. It’s pretty much the same as the tarball posted earlier.


--- fs_storage.py 2008-06-29 22:22:26 +1000
+++ fs_storage_ps3.py 2008-07-13 14:33:43 +1000
@@ -16,6 +16,7 @@
import mimetypes
mimetypes.init()
mimetypes.add_type('video/mp4', '.mp4')
+mimetypes.add_type('video/divx', '.avi')

from urlparse import urlsplit

@@ -556,6 +557,8 @@
'http-get:*:video/x-msvideo:*',
'internal:%s:video/avi:*' % self.server.coherence.hostname,
'http-get:*:video/avi:*',
+ 'internal:%s:video/divx:*' % self.server.coherence.hostname,
+ 'http-get:*:video/divx:*',
'internal:%s:video/quicktime:*' % self.server.coherence.hostname,
'http-get:*:video/quicktime:*'],
default=True)

Coherence - PS3

Recently i’ve been playing around with coherence. It’s pretty much an UPnP server that streams crap to your playstation3 or xbox 360. Problem is that even though the DivX/XviD format is supported in the newer firmwares, those files were displayed as ‘Unsupported Format’ in the PS3’s GameOS when being streamed over a default Coherence install. So, to remedy this i’ve taken the effort to research what caused the problem. You can do a diff if you want of these files, but everything is working fine for me (except for a lone MP3 file), but whatever.

Patched Coherence

Enjoy.

AC3Filter with 5.1ch AC3 Surround Sound - HDTV

Recently, i’ve been playing with my speakers and i’ve found out that they are not outputting *true* 5.1 audio, when it is available on the HD channel i’m watching.  The main issue i was running into was the fact that it was outputting CMSS surround audio (basically, it gets all of the channels, minus the center one, and then upmixes the left and right channels to create a center channel).  This had the unfortunate side effect of making music VERY loud during true 5.1 tv shows, and making dialogue impossible to understand (because the center channel is lost, and thats where voices are).  The only way to overcome this, that i have found so far was to use the following settings in ac3filter:

SPDIF - AC3 Encoder Bitrate: 320kbps, Use AC3 Encoder.

SPDIF Mode: SPDIF Wrapped.

SPDIF Passthrough: (untick all 3).  This allows you to reroute seperate channels on a 2.0/2.1 channel broadcast, and then upmix it to 5.1 (this is what i do).

That configuration was for standard definition channels, which allows upmixing and re-encodes audio.  I prefer that one because it sounds ‘crisper’ than using the raw mpeg audio.

This configuration is for true 5.1 programmes:

SPDIF - Disable AC3 Encoder.

SPDIF Mode: SPDIF Wrapped.

SPDIF Passthrough: AC3 Ticked.

That option will pass the untouched 5.1 channel audio from the tv station directly through to your speakers/decoder.  Using these configurations, you can easily switch between upmixed and original audio.

Before i finish, i just have to say that it taken me days on google to figure this problem out, so i hope it helps someone else in the future.

IIS ISAPI Plugin - Support for ~user Home Directories

Recently, I’ve started working with IIS and have discovered that there is a lack of a (working) plugin for IIS 6.0 that introduces support for ~user style home directories, so I found a plugin called "mapper" that supposedly worked for IIS 3/4/5, and I tried recompiling it to work for IIS 6.0 — no dice, so the only solution was to make one which did work with IIS 6.0.  This version is based upon a majority of the mapper code, however, some places have been improved for speed.  Code size of the finished dll was reduced from 40kb of the original, to 10.5/11kb with the new version.  Code is below (insert the standard stdafx.cpp etc).

   1: #include "stdafx.h"
   2: #include <windows.h>
   3: #include <httpfilt.h> 
   4: #include <shlwapi.h>
   5: #include <atlstr.h>
   6: #pragma comment(lib, "shlwapi.lib")
   7:  
   8: #define DEFAULT_BUFFER_SIZE 2048
   9: #define MAX_BUF 2048
  10: char szUsername[DEFAULT_BUFFER_SIZE];
  11: char *pszUsername = szUsername;
  12: const char szIndexFiles[8][13] = {"index.html","index.asp","index.aspx","default.htm","default.html","index.php","default.php","index.mspx"};
  13: char pszDebugEnabled[1024];
  14: char pszHomePath[1024];
  15: DWORD g_dwAppendBufferLen = 0;
  16:  
  17: bool DoInitialize(HMODULE hModule);
  18: DWORD DoUrlRewriting(HTTP_FILTER_CONTEXT *pfc, HTTP_FILTER_URL_MAP *pMap);
  19:  
  20: #ifdef _MANAGED
  21: #pragma managed(push, off)
  22: #endif
  23:  
  24: #define DLC_TRACE(x) DlcReportEventA(EVENTLOG_INFORMATION_TYPE, x)
  25:  
  26: void DlcReportEventA(WORD wType, LPCSTR pszMessage)
  27: {
  28:     if (pszDebugEnabled == NULL || pszDebugEnabled != "True") {
  29:         return;
  30:     } // we don’t write anything.
  31:  
  32:     HANDLE  hEventSource;
  33:     LPCTSTR  lpszStrings[1];
  34:  
  35:     lpszStrings[0] = (LPCTSTR)pszMessage;
  36:  
  37:     /* Get a handle to use with ReportEvent(). */
  38:     hEventSource = RegisterEventSourceA(NULL, "IISHome");
  39:     if (hEventSource != NULL)
  40:     {
  41:         /* Write to event log. */
  42:         ReportEventA(hEventSource, wType, 0, 0, NULL, 1, 0, (LPCSTR*) &lpszStrings[0], NULL);
  43:         DeregisterEventSource(hEventSource);
  44:     }
  45: }
  46:  
  47: BOOL GetRegistryKey (char *lpszConfigItem, char *outBuffer)
  48: {
  49:     char szRegistryResult[1024];
  50:     HKEY hKey;
  51:     BOOL bFoundItem = TRUE;
  52:     DWORD baseLen = 1024;
  53:     memset(szRegistryResult, 0, 1024);
  54:     char dbgInfo[DEFAULT_BUFFER_SIZE];
  55:     sprintf(dbgInfo, "DEBUG: Value Name: %s", lpszConfigItem);
  56:     DLC_TRACE(dbgInfo);
  57:  
  58:     if (RegOpenKeyEx(HKEY_LOCAL_MACHINE, "SOFTWARE\\BrentP\\IISHome", 0, KEY_READ, &hKey) != ERROR_SUCCESS) {
  59:         bFoundItem = FALSE;
  60:         DLC_TRACE("DEBUG: RegOpenKeyEx(…) failed to open key.");
  61:         return bFoundItem;
  62:     }
  63:  
  64:     if (bFoundItem == TRUE) {
  65:         /* Get Registry Value */
  66:         DLC_TRACE("DEBUG: Key found - ready for RegQueryValueEx(…)");
  67:         if (RegQueryValueEx(hKey, lpszConfigItem, NULL, NULL, (LPBYTE)szRegistryResult, &baseLen) != ERROR_SUCCESS) {
  68:             bFoundItem = FALSE;
  69:             DLC_TRACE("DEBUG: RegQueryValueEx(…) failed. No Value found.");
  70:         } else {
  71:             DLC_TRACE("DEBUG: upto strcpy(outBuffer, szRegistryResult)");
  72:             strcpy(outBuffer, szRegistryResult);
  73:         }
  74:     }
  75:  
  76:     RegCloseKey(hKey);
  77:     DLC_TRACE("DEBUG: RegCloseKey(hKey)");
  78:  
  79:     return bFoundItem;
  80: }
  81:  
  82: // End Registry Stuff
  83:  
  84: BOOL WINAPI __stdcall GetFilterVersion(HTTP_FILTER_VERSION *pVer)
  85: {
  86:     /* Specify the types and order of notification */
  87:  
  88:     pVer->dwFlags = (SF_NOTIFY_ORDER_HIGH | SF_NOTIFY_NONSECURE_PORT | 
  89:         SF_NOTIFY_SECURE_PORT | SF_NOTIFY_URL_MAP);
  90:  
  91:     pVer->dwFilterVersion = pVer->dwServerFilterVersion;
  92:     strcpy(pVer->lpszFilterDesc, (LPCSTR)"IISHome version 1.0");
  93:     DLC_TRACE("DEBUG: Started IISHome");
  94:  
  95:     // Now Init Registry Values
  96:     if (GetRegistryKey("HomePath", pszHomePath) == FALSE) {
  97:         strcpy(pszHomePath, "C:\\Users\\%s\\%s");
  98:     }
  99:     if (GetRegistryKey("DebugEnabled", pszDebugEnabled) == FALSE) {
 100:         strcpy(pszDebugEnabled, "False");
 101:     }
 102:  
 103:     char dbgInfo[DEFAULT_BUFFER_SIZE];
 104:     sprintf(dbgInfo, "DEBUG: HomePath: %s DebugEnabled: %s", pszHomePath, pszDebugEnabled);
 105:     DLC_TRACE(dbgInfo);
 106:  
 107:     return TRUE;
 108: }
 109:  
 110: BOOL FileExists(char *fileName) {
 111:     return GetFileAttributes(fileName) != INVALID_FILE_ATTRIBUTES;
 112: }
 113:  
 114: BOOL DirectoryExists(char *fileName) {
 115:     return GetFileAttributes(fileName) == FILE_ATTRIBUTE_DIRECTORY;
 116: }
 117:  
 118: DWORD DoUrlRewriting(HTTP_FILTER_CONTEXT *pfc, HTTP_FILTER_URL_MAP *pMap)
 119: {
 120:     char *s;
 121:     //unsigned int i; //allow conversion between size_t and int.
 122:     size_t i; // size_t is unsigned int. converting to int might cause loss of data.
 123:     DWORD dwRet = SF_STATUS_REQ_NEXT_NOTIFICATION;
 124:     /* Heres what we do:
 125:     1) Strip /~
 126:     2) Strip remaining /….. to get username alone.
 127:     3) Set Path to C:\Users\<username\
 128:     4) If any of the index files exist, strcat it on.
 129:     5) Now strcpy the physical path to the new path, and pass a return value.
 130:     */
 131:  
 132:     if (pfc == NULL || !pMap)
 133:     {
 134:         return dwRet;
 135:     }
 136:  
 137:     char szPhysicalPath[DEFAULT_BUFFER_SIZE];
 138:     DWORD cbBuf = DEFAULT_BUFFER_SIZE;
 139:  
 140:     if (!pszHomePath || pMap->pszURL[0] != ‘/’ || pMap->pszURL[1] != ‘~’) {
 141:         char dbgInfo[DEFAULT_BUFFER_SIZE];
 142:         sprintf(dbgInfo, "DEBUG: Could not match URL: %s", pMap->pszURL);
 143:         DLC_TRACE(dbgInfo);
 144:         return dwRet;
 145:     }
 146:  
 147:     // only match /~, not /bleh/~user.
 148:     char dbgInfo[DEFAULT_BUFFER_SIZE];
 149:     sprintf(dbgInfo, "DEBUG: URL Matched: %s", pMap->pszURL);
 150:     DLC_TRACE(dbgInfo);
 151:  
 152:     strcpy(szUsername, (pMap->pszURL)+2);
 153:     if (s = strchr(szUsername, ‘/’)) {
 154:         *s = (char)0; // replace it with a null character.
 155:         s++;
 156:         sprintf(szPhysicalPath, pszHomePath, szUsername, s);
 157:         char dbgInfo[DEFAULT_BUFFER_SIZE];
 158:         sprintf(dbgInfo, "DEBUG: Preliminary Physical Path: %s", szPhysicalPath);
 159:         DLC_TRACE(dbgInfo);
 160:     } else {
 161:         // we have nothing after the last /, so its http://server/~user/
 162:         sprintf(szPhysicalPath, pszHomePath, szUsername, "");
 163:         char dbgInfo[DEFAULT_BUFFER_SIZE];
 164:         sprintf(dbgInfo, "DEBUG: Preliminary Physical Path: %s", szPhysicalPath);
 165:         DLC_TRACE(dbgInfo);
 166:     }
 167:  
 168:     for (s = szPhysicalPath; *s; s++) {
 169:         if (*s == ‘/’) {
 170:             *s = ‘\\’;
 171:         } // replace / with a backslash
 172:     }
 173:  
 174:     DLC_TRACE("DEBUG: Replaced Slashes");
 175:  
 176:     // replace last character with a null value.
 177:     if (szPhysicalPath[i=strlen(szPhysicalPath)-1] == ‘\\‘) {
 178:         szPhysicalPath[i] = (char)0;
 179:         DLC_TRACE("DEBUG: Replaced Nulls");
 180:     }
 181:  
 182:     // FindFirstFile is inefficient, we use GetFileAttributes inside DirectoryExists.
 183:     if (szIndexFiles && DirectoryExists(szPhysicalPath)) {
 184:         DLC_TRACE("DEBUG: DirectoryExists(szPhysicalPath)=TRUE & szIndexFiles defined");
 185:         if (szPhysicalPath[i=strlen(szPhysicalPath)-1] != ‘\\’) {
 186:             strcat(szPhysicalPath, "\\"), i++;
 187:         }
 188:  
 189:         for (int j = 0; j < 9; j++) {
 190:             char szFName[2048];
 191:             strcpy(szFName, szPhysicalPath);
 192:             strcat(szFName, szIndexFiles[j]);
 193:  
 194:             if (FileExists(szFName)) {
 195:                 strcpy(szPhysicalPath, szFName);
 196:                 char dbgInfo[DEFAULT_BUFFER_SIZE];
 197:                 sprintf(dbgInfo, "DEBUG: Found: %s", szFName);
 198:                 DLC_TRACE(dbgInfo);
 199:                 break;
 200:             }
 201:         }
 202:     }
 203:  
 204:     if( strlen(szPhysicalPath) < pMap->cbPathBuff) {
 205:         strcpy(pMap->pszPhysicalPath, szPhysicalPath);
 206:         char dbgInfo[DEFAULT_BUFFER_SIZE];
 207:         sprintf(dbgInfo, "DEBUG: Final Path: %s", pMap->pszPhysicalPath);
 208:         DLC_TRACE(dbgInfo);
 209:     } else {
 210:         DlcReportEventA(EVENTLOG_INFORMATION_TYPE, (LPCSTR)"DEBUG: Couldnt remap URL - Not enough Memory");
 211:     }
 212:  
 213:     return dwRet;
 214: }
 215:  
 216: DWORD WINAPI __stdcall HttpFilterProc(HTTP_FILTER_CONTEXT *pfc, DWORD NotificationType, 
 217:                                       VOID *pvData)
 218: {
 219:     DWORD   status = SF_STATUS_REQ_NEXT_NOTIFICATION; 
 220:     switch (NotificationType)
 221:     {
 222:  
 223:     case SF_NOTIFY_URL_MAP:
 224:         status = DoUrlRewriting(pfc, (HTTP_FILTER_URL_MAP*)pvData);
 225:         break;
 226:     }
 227:     return status;
 228: }
 229:  
 230: bool DoInitialize(HMODULE hModule)
 231: {
 232:     return true;
 233: }
 234:  
 235: BOOL APIENTRY DllMain(HMODULE hModule, DWORD  ul_reason_for_call, LPVOID lpReserved)
 236: {
 237:     switch (ul_reason_for_call)
 238:     {
 239:     case DLL_PROCESS_ATTACH:
 240:         {
 241:             if (!DoInitialize(hModule))
 242:             {
 243:                 return FALSE;
 244:             }
 245:             break;
 246:         }
 247:     case DLL_PROCESS_DETACH:
 248:         {
 249:             g_dwAppendBufferLen = 0;
 250:         }
 251:         break;
 252:     }
 253:     return TRUE;
 254: }
 255: #ifdef _MANAGED
 256: #pragma managed(pop)
 257: #endif

 

Below is the registry file you’ll need to apply for it to work:

   1: REGEDIT4
   2:  
   3: [HKEY_LOCAL_MACHINE\SOFTWARE\BrentP\IISHome]
   4: "HomePath"="C:\\Users\\%s\\%s"
   5: "DebugEnabled"="False"

 

You’ll also need (because ISAPI requires some stuff to be exported) IISHome.def:

   1: LIBRARY    "IISHome"
   2:  
   3: ;DESCRIPTION  ‘IISHome Version 1.0′
   4:  
   5: EXPORTS
   6:     HttpFilterProc
   7:     GetFilterVersion

 

Once you’ve shoved all that into a visual studio project, you should be able to compile it — providing you’ve linked in Advapi32.lib for registry access and set the project to use a Multibyte character set, and only to use standard windows libraries (this can be done in the VS Project configuration dialogs).  Once you do this, the addon should compile with 0 warnings and errors, and then you can load it into IIS by right clicking on the website you want to load it in, then going to ISAPI Filters, then adding IISHome.  You will see any errors generated by IISHome in the application section of the error log (eventvwr.msc in Start -> Run).

You WILL also need to install the runtimes for whatever version of visual studio you compiled, for example, VS2005 or VS2008.  Enjoy.

This will probably be the last post for a while — again.

Alas, a new version of XMLTV Viewer.

A few days ago, i said a new build of XMLTV Viewer wouldn’t be coming for a while, but that was a build of the OLD version of XMLTV EPG Viewer.  This version has been rebuilt from the ground up in a few nights, is half the size of the old version, and contains about all the same features.  There are a few minor features that i decided to leave out because of their infrequent use, but all of the major features are back.

  • Grid View (plus, no ugly text box underneath grid view anymore.)
  • Now View (a new view which displays what tv shows are on now, or starting soon on all channels)
  • Search View (so you can search for tv shows)
  • and of course, our default listings view.
  • Better support for EPGStream.net feeds.
  • Cleaner display of data (in my opinion).
  • Stores settings in an xml file, not the registry (so they are much more portable).

There is a downside with this version though.  You will not be able to select a default feed from the installer.  That was removed because the previous version used the deployment wizard in Visual Studio, whereas the new version uses the WiX Project from Sourceforge for the installer framework.  As i said before, some features are not back in the new version (like channel logos, but they will be soon).  Some of the menu option’s also haven’t been linked up to their respective sections, so if that occurs, give me a yell.

Enjoy.

You can get it at: http://brentp.net/downloads/projects/xmltv/Installer.msi

The old version will remain up at the old url until i get all of the old features into the new one (minus the text colouring features and so on).

C# - XML File Handling

One thing that i’d have to say that sets the .NET framework apart from any other frameworks would be the fact that it has EXCELLENT XML File handling capabilities.  For example, take the following snippet (used in a program im submitting to a competition, i hope):

public int NewIndexRecovery()
{
    // this creates an entirely new index.  Pretty much the following:
    /* <Data>
     * </Data>
     */
    System.IO.File.Delete(Properties.Settings.Default.IndexDocument);
    XmlDocument xNewIndex = new XmlDocument();
    XmlNode xData = xNewIndex.CreateNode(XmlNodeType.Element, “Data”, null);
    XmlNode xComment = xNewIndex.CreateComment(“Index File - Created: ” + DateTime.Now.ToString());
    XmlNode xCommentTwo = xNewIndex.CreateComment(“V” + strVersion + “DATAFILE”);
    xData.AppendChild(xComment);
    xData.AppendChild(xCommentTwo);
    xNewIndex.AppendChild(xData);
    xNewIndex.Save(Properties.Settings.Default.IndexDocument);
    return 0;
}

 

That code pretty much creates a document similar to the following:

<Data>

    <!– Index File - Created 1/1/2008 12:38AM –!>

    <!– V2DATAFILE –!>

</Data>

 

The .NET framework handles all the pretty formatting as well, so you are actually able to read the files after you’re done with them.  Pretty cool huh?  Also, i have made an update to the XMLTV Viewer program, however, due to VS’s MSI maker thing complaining about missing bitmap files to build an installer, i am unable to upload it at the current time.  The new features in the current version are:

- EPGStream.net functionality (so you just enter in your epgstream id’s, and it’s ready to go.  The functionality can be turned on and off in the settings area, just remember that its a <LOT> buggy, so it may take some time to work.  Don’t try to download EPGStream feeds via the Settings -> XMLTV Location Downloader, because you will crash it, rather, use the main one in the main menu.  If you get a marquee progress bar (just continues going across, rather than an actual percentage), then its fetching an epgstream url.

- Bugfixes.

 EDIT: It’s just been uploaded at the usual url.  This will probably be the last build of it for some time (perhaps i might just recode the thing from scratch).

Download Link - XMLTV Viewer - LAST BUILD.

 

New(ish) Installer

I’ve just uploaded a debug build (so it’ll be a bit larger) of XMLTV EPG Viewer with a new installer that doesn’t bitch and moan at every turn.  I’ve also added a brief readme to the installer that describes how to initially set it up (as well as some credits to the people who contributed certain parts).

Same url as usual,

Enjoy.

More XMLTV Viewer updates

I’ve added some un-necessary eyecandy, as well as a ratings viewer (it shows you the rating of the tv show, ie: G/PG/M/etc).  The new eyecandy is a picture of the channels logo when you click on their name in the listview, looks pretty spiffy when you use it. Anyway, without further ado:

Download Link - XMLTV Viewer

XMLTV Reader - Universal Parsing

Latest version of XMLTV Reader Uploaded a minute or two ago

Improvements:

  • We only parse the file once now, so this will hopefully go a long way in reducing memory and cpu usage.
  • A Way to display extended information in grid view has been discovered, however, it is too taxing on the cpu. Might find a better method for the next version (using the Button/PictureBox/ButtonBase classes and overlaying that on the timeline)
  • Bug fixed where the whole timeline would get f’ed up if you went to any other view and switched back to the timeline view.
  • Removed old and unused code.
  • Improved the way that the extended info is displayed in the textbox below the table.  Before it was statically coded, now it determines which stuff has no information (ie: some shows might have a "Ep title" tidbit, whereas some shows might not).

Improvements for the next version:

  • I might actually read the xmltv spec <cough>
  • Get the extended info working in grid view.

Same link as usual. enjoy.