Uploading large files using HTML5, Microsoft MVC 3 and FineUploader

Standard

image_thumb.png

File sizes are getting larger (eg media files such as video) and increasingly web applications must be able to progressively go beyond the simple file input field.

Since the HTML5 spec supported the file api it is now possible to slice up very large files and send these progressively.

The FineUploader javascript library allows this to be done in a nice way.  Although the clientside example is complete, currently the MVC serverside example does not support file chunking.  (Although the Java example does)

Here’s an example I’ve created, based on the original that does.

I know that it would be better to update github (and perhaps I will) but for the time being It’s quicker to present it here.  I hope it helps someone and you are welcome to integrate it with FineUploaders example yourself.

Clientside code:

<script>
$(document).ready(function () {
$(‘#fine-uploader’).fineUploader({
request: {
endpoint: ‘Upload/UploadFile’
},
chunking: {
enabled: true,
chunksize: 100
},
debug: true,
autoupload: true
});
});
</script>

MVC Method:

[HttpPost]
public FineUploaderResult UploadFile(FineUpload upload)
{

// asp.net mvc will set extraParam1 and extraParam2 from the params object passed by Fine-Uploader
var dir = Server.MapPath(“~/App_Data/uploads”);

try
{

if (upload.IsMultipart())
{
// Save the part
var filePath = Path.Combine(dir, upload.PartUuid + “.” + upload.PartIndex);
upload.SaveAs(filePath,true);

// If last part do the merge
if(upload.PartIndex == (upload.TotalParts -1))
{
ulong bytesWritten = 0;
using (
var output = System.IO.File.OpenWrite(Path.Combine(dir, upload.OriginalFilename))
)
{
for (var i = 0; i < upload.TotalParts; i++) { // Open part file using ( var input = System.IO.File.OpenRead(Path.Combine(dir, upload.PartUuid + “.” + i)) ) { var buff = new byte[1]; while (input.Read(buff, 0, 1) > 0)
{
output.WriteByte(buff[0]);
bytesWritten++;
}
input.Close();
}
output.Flush();
}
output.Close();

if (bytesWritten != upload.FileSize)
{
throw new ApplicationException(“Upload Error. Please try again”);
}

// Clean up part files
for (var i = 0; i < upload.TotalParts; i++)
{
System.IO.File.Delete(Path.Combine(dir, upload.PartUuid + “.” + i));
}

}

}

}
else
{
var filePath = Path.Combine(dir, upload.Filename);
upload.SaveAs(filePath,true);
}

}
catch (Exception ex)
{
return new FineUploaderResult(false, error: ex.Message);
}

// the anonymous object in the result below will be convert to json and set back to the browser
return new FineUploaderResult(true, new { extraInformation = 12345 });
}
FineUploader.cs:

using System.IO;
using System.Web.Mvc;

namespace FineUploader
{
[ModelBinder(typeof(ModelBinder))]
public class FineUpload
{

private const string FILENAME_PARAM = “qqfile”;
private const string PART_INDEX_PARAM = “qqpartindex”;
private const string FILE_SIZE_PARAM = “qqtotalfilesize”;
private const string TOTAL_PARTS_PARAM = “qqtotalparts”;
private const string UUID_PARAM = “qquuid”;
private const string PART_FILENAME_PARAM = “qqfilename”;
private const string BLOB_NAME_PARAM = “qqblobname”;

public string Filename { get; set; }
public Stream InputStream { get; set; }
public int PartIndex { get; set; }
public int TotalParts { get; set; }
public string OriginalFilename { get; set; }
public string PartUuid { get; set; }
public ulong FileSize { get; set; }
public bool IsMultipart()
{
return (TotalParts > 0);
}
public void SaveAs(string destination, bool overwrite = false, bool autoCreateDirectory = true)
{
if (autoCreateDirectory)
{
var directory = new FileInfo(destination).Directory;
if (directory != null) directory.Create();
}

using (var file = new FileStream(destination, overwrite ? FileMode.Create : FileMode.CreateNew))
InputStream.CopyTo(file);
}

public class ModelBinder : IModelBinder
{
public object BindModel(ControllerContext controllerContext, ModelBindingContext bindingContext)
{
var request = controllerContext.RequestContext.HttpContext.Request;
var formUpload = request.Files.Count > 0;

// find filename
var xFileName = request.Headers["X-File-Name"];
var qqFile = request[FILENAME_PARAM];
var qqpartindex = request[PART_INDEX_PARAM];

var formFilename = formUpload ? request.Files[0].FileName : null;

var upload = new FineUpload
{
Filename = xFileName ?? qqFile ?? formFilename,
InputStream = formUpload ? request.Files[0].InputStream : request.InputStream,

};
int someval;
if (int.TryParse(qqpartindex, out someval))
{
upload.PartIndex = someval;
upload.OriginalFilename = request[PART_FILENAME_PARAM];
upload.TotalParts = int.Parse(request[TOTAL_PARTS_PARAM]);
upload.PartUuid = request[UUID_PARAM];
upload.FileSize = ulong.Parse(request[FILE_SIZE_PARAM]);
}

return upload;
}
}

}
}

This code is supplied as is, for informational purposes only with no warranties or statements about being fit for purpose and stuff.

Its time. I’m making a stand. MyNamingConventionManifesto.

Standard

Variable names, filenames and other technical names.  Hungarian notion, abbreviations, verbosity, camel case and mixed case.  How many of my projects have used mixtures of these conventions?  How often do I forget what abbrev I usd????

?!It_never_ends!?   

no-longer   

I-AM-SICK-OF-IT!

_this_has_to_stop

<rant>It is time to do away with inconsistency forever.  </rant>

I am happy to comply with the conventions of a language (eg in .net mixed case is used for public attributes and properties whereas camel case is used for parameters) or the conventions of an existing project, but if there isn’t any, MyNamingConventionManifesto comes into play.

The decisions are around:

CASE (Upper, Lower, Mixed or Camel Case) 

DELIMITER (None, Minus, Underscore or Space)

PREFIX

SUFFIX

So, I hereby declare that my naming convention will not use prefixes or suffixes, neither will it use delimiters.  In short, this is:

MyNamingConventionManifesto 

 

ThankyouThatWillBeAll

 

Simple object dumping extension method for CSharp

Standard

Here’s a useful little extension method I created to dump fields and properties to the console of an object in CSharp.  

It demonstrates simple reflection and extension method syntax.

 

 

public static class ObjectExtensions
{
public static void DumpProperties(this object obj)
{
Console.WriteLine(obj.GetType().Name);
Console.WriteLine("{");
foreach (var fld in obj.GetType().GetFields())
{
Console.WriteLine(String.Format("{0} = \"{1}\"", fld.Name, fld.GetValue(obj)));
}
foreach (var prop in obj.GetType().GetProperties())
{
if(prop.CanRead) Console.WriteLine(String.Format("{0} = \"{1}\"", prop.Name,prop.GetValue(obj, null)));
}
Console.WriteLine("}");
}
}

 

 

 

Creating a an instance in EC2 with .net and passing parameters

Standard

When starting an Amazon EC2 instance, you can pass something called “user metadata”.  This can be a file or values.  These values are not stored in an environmental variable or file on the new instance, but can instead be retrieved by doing a HTTP Get to a “special” IP address to retrieve those values.

For example:

1.  Create the instance

RunInstancesResponse response = Client.RunInstances(new RunInstancesRequest()
  .WithImageId(ami_id)
  .WithInstanceType(instance_type)
  .WithKeyName(YOUR_KEYPAIR_NAME)
  .WithMinCount(1)
  .WithMaxCount(max_number_of_instances)
  .WithUserData(Convert.ToBase64String(Encoding.UTF8.GetBytes(bootScript.Replace("\r", ""))))
);

2.  In a startup script on the instance, make a call like this:

GET http://169.254.169.254/latest/user-data
1234,fred,reboot,true | 4512,jimbo, | 173,,,

See the links below for further info

 

 

 

 

 

Links

http://stackoverflow.com/questions/7420368/how-to-start-an-amazon-ec2-instance-programmatically-in-net

http://docs.amazonwebservices.com/AWSEC2/latest/UserGuide/AESDG-chapter-instancedata.html

What every phonegap developer on windows phone 7 needs

Standard

 

Error logging!  An unfortunate side effect of developing apps in Javascript in mobile IE9 is that there doesn’t appear to be any error dialogs.

Try the following Javascript:

 

// provide our own console if it does not exist, huge dev aid!
if (typeof window.console == “undefined”) {
window.console = { log: function (str) { window.external.Notify(str); } };
}

// output any errors to console log, created above.
window.onerror = function (msg,url,linenumber) {
console.log(“Error ::” + msg + ” in ” + url + ” at line ” + linenumber);
};

console.log(“Installed console ! “);

This also gives you a “console.log” for free.

 

Enjoy!

 

 

Aspect Oriented Programming in Javascript

Standard

 

In designing javascript for modularity and “separation of concerns”, I was pleasantly surprised to discover a way of implementing AOP (Aspect Oriented Programming) in Javascript.

 

AOP can be useful for automatically “wrapping” methods to provide tracing, error logging or caching.

In this example, we are “wrapping” a method to show an alert when a method starts and finishes:

$.aop.around({ target: homelinksModel, method: 'get' }
 , function (invocation) {
alert("Method Call");
 
var ret = return invocation.proceed(); // This line calls the original method
alert("After Call");
return ret;
});

I used it for caching using Lawnchair  (an abstraction around local storage on client browsers).  It is designed to cache the result of data calls to the server.

$.aop.around({ target: homelinksModel, method: 'get' }
 , function (invocation) {
var lc = new Lawnchair(function () { });
var keyName = 'homelinksModel' + invocation.method;
var rec;
lc.get(keyName, function (ret) {
if (ret == null) {
    res = invocation.proceed()
    lc.save({ key: keyName, value: res });
    rec = res;
 }
 else {
 rec = ret.value;
 }
});
return rec;
}); // </aop>

I’m still learning Lawnchair, so there’s probably better ways of implementing this, but I thought a “useful” example might be helpful.

Peter Chung has an excellent article about Jquery AOP if you’d like to learn more.

There’s not alot of recent “action” around AOP in the Javascript space (Google Search).   I suspect that either some of it’s functionality is automatically part of jQuery or that the Javascript world isn’t mature enough yet to care much.    Perhaps it’s the former.

Over the past few days I’ve had a marvelous time implementing the MVVM pattern in Javascript.  I’m discovering all my favourite technologies are there, such as object databases (Lawnchair), binding (Knockout) and much more.  I hope to be blogging more about some of these soon.

Links

 

 

Returning data from an MVC method with a small cute ActionResult

Standard

image

In order return data from an MVC method, Microsoft provide a useful ActionResult object called JSONResult.  JSONResult exposes a convenient Data property that you can use to serialize an object to JSON.

Here’s how it can be used:

var res = new JsonResult();
res.JsonRequestBehavior = JsonRequestBehavior.AllowGet;
res.Data = someObject;
return res;

Pretty easy to use.   Unfortunately there’s no matching XMLResult class if you want to return a scrap of XML instead.  In the MVCContrib project on codeplex there’s an XMLResult, but it’s event signature is slightly different.

I’ve modified this class to provide the same signature. 

Here’s the usage:

var res = new XmlResult();
res.Data = someObject;
return res;

Here’s the class:

using System.Web.Mvc;
using System.Xml.Serialization;

namespace Sample
{
    /// <summary>
    /// Action result that serializes the specified object into XML and outputs it to the response stream.
    /// </summary>
    public class XmlResult : ActionResult
    {
        private object _objectToSerialize;
        private XmlAttributeOverrides _xmlAttribueOverrides;

        /// <summary>
        /// Creates a new instance of the XmlResult class.
        /// </summary>
        /// <param name="objectToSerialize">The object to serialize to XML.</param>
        public XmlResult()
        {
      
        }

        /// <summary>
        /// Creates a new instance of the XmlResult class.
        /// </summary>
        /// <param name="objectToSerialize">The object to serialize to XML.</param>
        public XmlResult(object objectToSerialize)
        {
            _objectToSerialize = objectToSerialize;
        }

        /// <summary>
        /// Creates a new instance of the XMLResult class.
        /// </summary>
        /// <param name="objectToSerialize">The object to serialize to XML.</param>
        /// <param name="xmlAttributeOverrides"></param>
        public XmlResult(object objectToSerialize, XmlAttributeOverrides xmlAttributeOverrides)
        {
            _objectToSerialize = objectToSerialize;
            _xmlAttribueOverrides = xmlAttributeOverrides;
        }

        /// <summary>
        /// The object to be serialized to XML.
        /// </summary>
        public object Data
        {
            get
            { return _objectToSerialize; }
            set
            {
                _objectToSerialize = value;
            }
        }

        /// <summary>
        /// Serialises the object that was passed into the constructor to XML and writes the corresponding XML to the result stream.
        /// </summary>
        /// <param name="context">The controller context for the current request.</param>
        public override void ExecuteResult(ControllerContext context)
        {
            if (_objectToSerialize != null)
            {
                var xs = (_xmlAttribueOverrides == null) ?
                    new XmlSerializer(_objectToSerialize.GetType()) :
                    new XmlSerializer(_objectToSerialize.GetType(), _xmlAttribueOverrides);
                context.HttpContext.Response.ContentType = "text/xml";
                xs.Serialize(context.HttpContext.Response.Output, _objectToSerialize);
            }
        }
    }
}

Data Uris–A way of embedding images in your html page or UIWebView

Standard

image

Currently I’m investigating ways of using  UIWebView to render a user interface.  

I’d like to re-purpose some server side html form rendering logic so that it can be resused on the iPhone, Android and Windows Phone. 

Modern forms incorporate more than simple form elements however.  They may include data, such as images.

By default, the UIWebview supports images in the “bundle” and images in the documents area. 

What if I want to create images in memory, then display these in the HTML?

An little known feature of many modern web browsers (including Webkit) is datauris.  A datauri allows data to be embedded in a webpage where a url would usually be.

Here’s an example:

<img src="data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAUA
AAAFCAYAAACNbyblAAAAHElEQVQI12P4//8/w38GIAXDIBKE0DHxgljNBAAO
9TXL0Y4OHwAAAABJRU5ErkJggg==" alt="Red dot" />
There are disadvantages however, which are the obvious effect this will have on caching and pagesize.
The Stackflow article Display Local UI Image On a UIWebView shows how to this, replicated here for your convenience.
This code demonstrates reading a file from the local bundle, then embedding into the background image, in the html dom of a UIWebView:

#import "NSString+DataURI.h"

#import "NSData+Base64.h"

...

-(void)webViewDidFinishLoad:(UIWebView *)webView

{

    NSString *imgPath = [[NSBundle mainBundle] pathForResource:@"image" ofType:@"png"];

    NSData *imgData = [NSData dataWithContentsOfFile:imgPath];

    NSString *imgB64 = [[imgData base64Encoding] pngDataURIWithContent];

    NSString *javascript = [NSString stringWithFormat:@"document.body.style.backgroundImage='url(%@)';", imgB64];

    [webView stringByEvaluatingJavaScriptFromString:javascript];

}

This code uses the following utility functions that provide the base 64 encoding support:

NSData+Base64.h

@interface NSData (Base64)

+ (NSData *)dataWithBase64EncodedString:(NSString *)string;

- (id)initWithBase64EncodedString:(NSString *)string;

- (NSString *)base64Encoding;

- (NSString *)base64EncodingWithLineLength:(unsigned int) lineLength;

@end

NSData.Base64.m

#import "NSData+Base64.h"

static char encodingTable[64] = {

'A','B','C','D','E','F','G','H','I','J','K','L','M','N','O','P',

'Q','R','S','T','U','V','W','X','Y','Z','a','b','c','d','e','f',

'g','h','i','j','k','l','m','n','o','p','q','r','s','t','u','v',

'w','x','y','z','0','1','2','3','4','5','6','7','8','9','+','/' };

@implementation NSData (VQBase64)

- (id)initWithString:(NSString *)string {

    if ((self = [super init])) {

        [self initWithBase64EncodedString:string];

    }

    return self;

}

+ (NSData *) dataWithBase64EncodedString:(NSString *) string {

    return [[[NSData allocWithZone:nil] initWithBase64EncodedString:string] autorelease];

}

- (id) initWithBase64EncodedString:(NSString *) string {

    NSMutableData *mutableData = nil;

    if( string ) {

        unsigned long ixtext = 0;

        unsigned long lentext = 0;

        unsigned char ch = 0;

        unsigned char inbuf[4], outbuf[3];

        short i = 0, ixinbuf = 0;

        BOOL flignore = NO;

        BOOL flendtext = NO;

        NSData *base64Data = nil;

        const unsigned char *base64Bytes = nil;

        // Convert the string to ASCII data.

        base64Data = [string dataUsingEncoding:NSASCIIStringEncoding];

        base64Bytes = [base64Data bytes];

        mutableData = [NSMutableData dataWithCapacity:[base64Data length]];

        lentext = [base64Data length];

        while( YES ) {

            if( ixtext >= lentext ) break;

            ch = base64Bytes[ixtext++];

            flignore = NO;

            if( ( ch >= 'A' ) && ( ch <= 'Z' ) ) ch = ch - 'A';

            else if( ( ch >= 'a' ) && ( ch <= 'z' ) ) ch = ch - 'a' + 26;

            else if( ( ch >= '0' ) && ( ch <= '9' ) ) ch = ch - '0' + 52;

            else if( ch == '+' ) ch = 62;

            else if( ch == '=' ) flendtext = YES;

            else if( ch == '/' ) ch = 63;

            else flignore = YES;

            if( ! flignore ) {

                short ctcharsinbuf = 3;

                BOOL flbreak = NO;

                if( flendtext ) {

                    if( ! ixinbuf ) break;

                    if( ( ixinbuf == 1 ) || ( ixinbuf == 2 ) ) ctcharsinbuf = 1;

                    else ctcharsinbuf = 2;

                    ixinbuf = 3;

                    flbreak = YES;

                }

                inbuf [ixinbuf++] = ch;

                if( ixinbuf == 4 ) {

                    ixinbuf = 0;

                    outbuf [0] = ( inbuf[0] << 2 ) | ( ( inbuf[1] & 0x30) >> 4 );

                    outbuf [1] = ( ( inbuf[1] & 0x0F ) << 4 ) | ( ( inbuf[2] & 0x3C ) >> 2 );

                    outbuf [2] = ( ( inbuf[2] & 0x03 ) << 6 ) | ( inbuf[3] & 0x3F );

                    for( i = 0; i < ctcharsinbuf; i++ )

                        [mutableData appendBytes:&outbuf[i] length:1];

                }

                if( flbreak )  break;

            }

        }

    }

    self = [self initWithData:mutableData];

    return self;

}

#pragma mark -

- (NSString *) base64Encoding {

    return [self base64EncodingWithLineLength:0];

}

- (NSString *) base64EncodingWithLineLength:(unsigned int) lineLength {

    const unsigned char     *bytes = [self bytes];

    NSMutableString *result = [NSMutableString stringWithCapacity:[self length]];

    unsigned long ixtext = 0;

    unsigned long lentext = [self length];

    long ctremaining = 0;

    unsigned char inbuf[3], outbuf[4];

    unsigned short i = 0;

    unsigned short charsonline = 0, ctcopy = 0;

    unsigned long ix = 0;

    while( YES ) {

        ctremaining = lentext - ixtext;

        if( ctremaining <= 0 ) break;

        for( i = 0; i < 3; i++ ) {

            ix = ixtext + i;

            if( ix < lentext ) inbuf[i] = bytes[ix];

            else inbuf [i] = 0;

        }

        outbuf [0] = (inbuf [0] & 0xFC) >> 2;

        outbuf [1] = ((inbuf [0] & 0x03) << 4) | ((inbuf [1] & 0xF0) >> 4);

        outbuf [2] = ((inbuf [1] & 0x0F) << 2) | ((inbuf [2] & 0xC0) >> 6);

        outbuf [3] = inbuf [2] & 0x3F;

        ctcopy = 4;

        switch( ctremaining ) {

            case 1:

                ctcopy = 2;

                break;

            case 2:

                ctcopy = 3;

                break;

        }

        for( i = 0; i < ctcopy; i++ )

            [result appendFormat:@"%c", encodingTable[outbuf[i]]];

        for( i = ctcopy; i < 4; i++ )

            [result appendString:@"="];

        ixtext += 3;

        charsonline += 4;

        if( lineLength > 0 ) {

            if( charsonline >= lineLength ) {

                charsonline = 0;

                [result appendString:@"\n"];

            }

        }

    }

    return [NSString stringWithString:result];

}

@end

  

NSString+DataURI.h

#import <Foundation/Foundation.h>

@interface NSString(DataURI)

- (NSString *) pngDataURIWithContent;

- (NSString *) jpgDataURIWithContent;

@end

NSString+DataURI.m

#import "NSString+DataURI.h"

@implementation NSString(DataURI)

- (NSString *) pngDataURIWithContent;

{

    NSString * result = [NSString stringWithFormat: @"data:image/png;base64,%@", self];

    return result;

}

- (NSString *) jpgDataURIWithContent;

{

    NSString * result = [NSString stringWithFormat: @"data:image/jpg;base64,%@", self];

    return result;

}

@end

  

 

Setting the default browser in Visual Studio 2010 in an MVC Application

Standard

image

To set the default browser used for debugging in an ASP.Net web application you need to bring up the “Browse With…” option on an aspx web page.  In a default MVC application you won’t have this option.

To do this temporarily add a “default.aspx” form to the root of your project, then the “Browse With…” context menu option will appear.  Choose your favourite browser, then remove the “default.aspx”.

Updating Twitter via SuperTweet using VBA

Standard

image

In my previous article I showed how to update a twitter status, prior to the twitter decision to remove the basic authentication api.  This article shows a way of getting around this.  The SuperTweet service allows you to continue to use basic authentication.  This is particularly useful for non-visual processes that use VBA.

I am currently using Twitter to update a private tweet-stream of my activities (such as sending email) from an Outlook 2010 macro.  This stream is then fed into the iPhone application Momento, which provides me with an “autodiary” based on my various twitter feeds and online services.

To use the SuperTweet, there are just three steps:

  1. Sign In with OAuth to the SuperTweet.net  (Sign-Up/Sign-In Now)
  2. Create a password for your applications to use with the SuperTweet.net API when they want to Tweet: Learn more.
  3. Change your application to use http://api.supertweet.net instead of Twitter.com.  See the sample below on how to do this using WinHttp in VBA. ( Learn more.)

The VBA code below will allow you to do an update to Twitter.  The UrlEncode implementation came from this StackOverflow post.

Function PostToTwitter(statusUpdate As String, username As String, password As String) As Boolean
    Const HTTPREQUEST_SETCREDENTIALS_FOR_SERVER = 0 
    On Error GoTo error_handler
   
    Dim WinHttpReq As New WinHttpRequest
 
     ' Assemble an HTTP Request.
    WinHttpReq.Open "POST", _
      "http://api.supertweet.net/1/statuses/update.xml", False
    WinHttpReq.SetCredentials username, password, HTTPREQUEST_SETCREDENTIALS_FOR_SERVER
    ' Send the HTTP Request.
    WinHttpReq.SetRequestHeader "Content-Type", "application/x-www-form-urlencoded"
    WinHttpReq.Send "status=" & URLEncode(statusUpdate)
    If WinHttpReq.Status <> 200 Then
      
       GoTo error_handler
    End If
    DoEvents
    Debug.Print "Posted - " & statusUpdate
   
    PostToTwitter = True
   
    Exit Function

error_handler:

    PostToTwitter = False
   
    Exit Function
       
End Function

Public Function URLEncode( _
   StringToEncode As String, _
   Optional UsePlusRatherThanHexForSpace As Boolean = False _
) As String

  Dim TempAns As String
  Dim CurChr As Integer
  CurChr = 1

  Do Until CurChr - 1 = Len(StringToEncode)
    Select Case Asc(Mid(StringToEncode, CurChr, 1))
      Case 48 To 57, 65 To 90, 97 To 122
        TempAns = TempAns & Mid(StringToEncode, CurChr, 1)
      Case 32
        If UsePlusRatherThanHexForSpace = True Then
          TempAns = TempAns & "+"
        Else
          TempAns = TempAns & "%" & Hex(32)
        End If
      Case Else
        TempAns = TempAns & "%" & _
          Right("0" & Hex(Asc(Mid(StringToEncode, _
          CurChr, 1))), 2)
    End Select

    CurChr = CurChr + 1
  Loop

  URLEncode = TempAns
End Function