Lookup in db or in code (enum)?
Maybe this is a solved problem already, but I couldn’t figure it out until today. With massive usage of NHibernate, lookup in db or in code always appear as a puzzle to me. For example, given a lookup dictionary like mime type: text/plain, text/html, image/gif and other stuff. It make sense to save all those data into a table then enable foreign key on all referenced tables. But, in code, those mime type id itself will appear like magic number, I don’t likes this.
So, creating an enum contains all items solve the magic number issue, partly, and introduce another sync concern, what if data in db and enum in code out of sync?
This post demos a brilliant idea of creating enum on the fly, but mime value has slash in it, this solution doesn’t work. Also, wrapper class around enum seems unnecessary.
Based on the suggestion from this post. here comes my solution,
public enum MimeType
{
[Description("application/octet-stream")] ApplicationOctetStream,
[Description("text/plain")] TextPlain = 1,
[Description("text/html")] TextHtml,
[Description("application/pdf")] ApplicationPdf,
[Description("application/vnd.ms-excel")] ApplicationVndMsExcel,
[Description("image/gif")] ImageGif,
[Description("image/jpeg")] ImageJpeg,
[Description("application/rtf")] ApplicationRtf,
[Description("application/zip")] ApplicationZip,
[Description("application/msword")] ApplicationMsword,
[Description("application/mspowerpoint")] ApplicationMspowerpoint
}
/// <summary>
/// Extension methods container for enum used to check synchronization between enum in code and data in db.
/// see usage in should_fetch_all_mime_types() of EmailQueueRepositoryTest class
/// </summary>
public static class MimeTypeEx
{
public static MimeType ToMimeTypeValue(this string value)
{
foreach (FieldInfo fi in typeof (MimeType).GetFields())
{
if (!fi.IsStatic)
{
continue;
}
object[] attrs = fi.GetCustomAttributes(typeof (DescriptionAttribute), false);
if (attrs == null || attrs.Length <= 0)
{
continue;
}
var descr = (DescriptionAttribute) attrs[0];
if (0 == string.Compare(value.Trim(), descr.Description))
// in case varchar type. trim it before compare.
{
return (MimeType) fi.GetValue(null);
}
}
throw new InvalidDataException(value);
//return MimeType.Unknown;
}
public static string ToMimeTypeString(this MimeType value)
{
foreach (FieldInfo fi in typeof (MimeType).GetFields())
{
if (!fi.IsStatic || (MimeType) fi.GetValue(null) != value)
{
continue;
}
object[] attrs = fi.GetCustomAttributes(typeof (DescriptionAttribute), false);
if (attrs == null || attrs.Length <= 0)
{
continue;
}
var descr = (DescriptionAttribute) attrs[0];
return descr.Description;
}
return string.Empty;
}
}
/// <summary>
/// Used to read mime type from db, then we can check synchronization with defined enum mime type.
/// </summary>
public class MimeTypeInDb
{
public virtual int Id { get; set; }
public virtual string Name { get; set; }
public virtual bool BinaryInd { get; set; }
}
Test code, similar functionality should be done in app start or somewhere static ctor to ensure the synchronization.
[Test]
public void should_fetch_all_mime_types()
{
var results = _repository.FetchAll<MimeTypeInDb>();
Assert.That(results.Count(), Is.GreaterThan(0));
foreach (MimeTypeInDb mimeTypeInDb in results)
{
Console.WriteLine(mimeTypeInDb.Name.ToMimeTypeValue()); // will throw invalid data exception if enum not found.
}
foreach (MimeType mimeType in Enum.GetValues(typeof(MimeType)))
{
Console.WriteLine(mimeType.ToMimeTypeString());
if (! results.Any(x => x.Name.Trim() == mimeType.ToMimeTypeString()))
{
Assert.Fail(mimeType.ToMimeTypeString() +" not found in db.");
}
}
}
About mapping the enum to int in FluentNHibernate:
Map(x => x.MimeType).Column("mime_type_id").CustomType<int>();
Cloud means surprise
I have been looking for an opportunity to work on queue base project after finishing Udi Dahan’s Advanced distributed system design course. Because most of my projects are running on Linux OS where NSerivceBus is not applicable, using Amazon SQS became a affordable work around for now.
Problem to solve
My iClip iOS app recently encountered unexpected popular requests for unknown reason (listed somewhere on an iOS app category site?). Even I only grant every new client only 3 free requests, with the huge amount of daily downloads, my server becomes slow and unstable gradually.
I could keep bumping up the memory on server, but I don’t like this solution at all. First, server hosting bill will go up dramatically; second, hosting server memory still has a max limitation of 4GB. Someday the client requests will eventually break this ceiling, then what?
Solution
I decided to move my request model to queue based architecture based on Amazon’s SQS service. Rewriting app didn’t take very long. After a week of production experience, here are some gotcha I’ve learned.
- Message won’t be immediately available after send. Amazon actually state this delay will be up to 60 seconds on SQS admin console.
- Deleted message might still be visible in queue. Explanation from Amazon is:
It is possible you will receive a message even after you have deleted it. This might happen on rare occasions if one of the servers storing a copy of the message is unavailable when you request to delete the message. The copy remains on the server and might be returned to you again on a subsequent receive request. You should create your system to be idempotent so that receiving a particular message more than once is not a problem.- For the similar reason as above, queue processor should really deal with duplicated request very carefully. For me, I use sendtimestamp as the identity of message.
- Receiving message must set the visibility time out to a value greater than zero, to allow this message to be deleted successfully afterwards. (To be confirmed)
Re-configure dreamhost evniorment to support Amazon WebService
I was trying to make my dreamhost to support Amazon WebService (aws), the first problem I encountered is, missing libxslt.
Following the instruction here, I managed to install libxslt to my custom location.
Watch out, the link on http://xmlsoft.org/XSLT/downloads.html is actually pointing to libxml2 which doesn’t include libxslt. You want to find the correct link from ftp://xmlsoft.org/libxslt/
The next problem is openssl support missing from ruby. I had to re-configure ruby to make it openssl ready. Instruction can be found here.
All aws samples passed, then.
With one minor issue left, I got warning when running ruby:
Invalid gemspec in [/home/me/.gems/specifications/openssl-extensions-1.2.0.gemspec]: invalid date format in specification: “2011-11-03 00:00:00.000000000Z”
Hope I can figure this out later.
- filter type set to log4net.Filter.StringMatchFilter
- each filter section can only have one stringToMatch element
- end with a “log4net.Filter.DenyAllFilter” filter to switch from the default “accept all unless instructed otherwise” filtering behavior to a “deny all unless instructed otherwise” behavior.
<?xml version="1.0" encoding="utf-8"?>
<log4net debug="false">
<appender name="SmtpAppender_WebTeam" type="log4net.Appender.SmtpAppender">
<to value="goodguy@me.com" />
<from value="log4net@me.ca" />
<subject value="!!!Web Exception happened!!!" />
<smtpHost value="mail.server.ds" />
<bufferSize value="512" />
<lossy value="true" />
<evaluator type="log4net.Core.LevelEvaluator">
<threshold value="ERROR"/>
</evaluator>
<layout type="log4net.Layout.PatternLayout">
<conversionPattern value="%newline%date [%thread] %-5level %logger [%property{NDC}] - %message%newline%newline%newline" />
</layout>
<filter type="log4net.Filter.StringMatchFilter">
<stringToMatch value="RequestManagement" />
</filter>
<filter type="log4net.Filter.StringMatchFilter">
<stringToMatch value="VineOnLine" />
</filter>
<filter type="log4net.Filter.DenyAllFilter" />
</appender>
<appender name="SmtpAppender_VSE" type="log4net.Appender.SmtpAppender">
<to value="nice@me.ca" />
<from value="log4net@mr.ca" />
<subject value="!!!VSE Exception happened!!!" />
<smtpHost value="mail.server.ds" />
<bufferSize value="512" />
<lossy value="true" />
<evaluator type="log4net.Core.LevelEvaluator">
<threshold value="ERROR"/>
</evaluator>
<layout type="log4net.Layout.PatternLayout">
<conversionPattern value="%newline%date [%thread] %-5level %logger [%property{NDC}] - %message%newline%newline%newline" />
</layout>
<filter type="log4net.Filter.StringMatchFilter">
<stringToMatch value="VSE" />
</filter>
<filter type="log4net.Filter.DenyAllFilter" />
</appender>
<root>
<level value="INFO" />
<appender-ref ref="SmtpAppender_WebTeam" />
<appender-ref ref="SmtpAppender_VSE" />
</root>
<logger name="NHibernate">
<level value="ERROR"/>
</logger>
</log4net>
NHibernate Session.Query ignore Fetch Join?
We set up child collection relationship to Fetch as Join in mapping, using fluentNHibernate:
HasMany( x => Details ).Fetch.Join();
It works fine at least for Session.Get<T>(id), we can see only one joined query instead of two separated ones.
But it seems Session.Query<T>() keeps ignoring this join fetch setting, we always got two separated selects when using Session.Query<T>. A Bug?
Share static html in MVC views
PartialView can be used to share page content, but our shared content are purely static and would like to share it among multiple web apps, doesn’t have to be MVC apps.
We have tried SSI (Server side include) on IIS, but it turns out it doesn’t work in Razor view. Spuriously, many people said SSI is dead, and by default IIS7 doesn’t turn SSI on, here is the doc how to install and configure SSI on IIS7. install it, configure it (if your included file is not named as shtm)
For now the best solution I found is putting this code in Razor view.
@MvcHtmlString.Create(File.ReadAllText(Server.MapPath(“/_shared/footer.htm”)))
Make method parameter strongly typed without if/switch
Given a class method can generate report based on the type passed in from method parameter, an easy way to make parameter strongly typed is using enum:
class ReportGenerator
{
string _reportTypeInString;
enum ReportType
{
PDF = 0,
EXCEL,
HTML
}
void GetReport(ReportType reportType)
{
switch (reportType)
{
case ReportType.PDF:
_reportTypeInString = "PDF";
break;
case ReportType.EXCEL:
_reportTypeInString = "EXCEL";
break;
case ReportType.HTML:
_reportTypeInString = "HTML40";
break;
}
// logic to generate real report...
}
}
The problem here is: enum type in dotnet only supports integer, converting to string has to be done somewhere usually in a place different than the type define area, maintenance is problematic. Besides, the evil if/switch code smell.
Here is my way to do the same job:
public class ReportFormat
{
public string ReportFormatString { get; private set; }
public static ReportFormat HTML32 = new ReportFormat("HTML3.2");
public static ReportFormat HTML40 = new ReportFormat("HTML4.0");
public static ReportFormat EXCEL = new ReportFormat("EXCEL");
public static ReportFormat CSV = new ReportFormat("CSV");
public static ReportFormat PDF = new ReportFormat("PDF");
public static ReportFormat MHTML = new ReportFormat("MHTML");
ReportFormat(string reportFormatString)
{
ReportFormatString = reportFormatString;
}
}
//in my report generator class:
public FileContentResult GetReport(string reportName, ReportFormat reportFormat, IList<ParameterValue> parameters)
{...}
// how to use
var fileContentResult = reportingServiceProxy.GetReport(ReportName, ReportingServiceProxy.ReportFormat.HTML40, parameters);
Work amount is almost same, the type maintenance is in same place, the biggest selling point, no evil if/switch statement anymore.
Which design pattern is used here?
How to unit-test class has dependency on HttpResponse
When getting report from reporting service through asmx web service, the proxy class needs to set some custom header to HttpResponse, work can be done, but unit-test it is not easy. Google result indicates that we can take advantage of HttpResponseBase and HttpResponseWrapper from System.Web and System.Web.Abstraction.
The class diagram looks like this:
Instead of directly depending on HttpContext.Current.Response, we need to change the dependency to HttpResponseBase class, from which both HttpResponseWrapper (in production) and HttpResponseFake (in unit-test) are derived.
Code example:
public class ReportingServiceProxy
{
private readonly HttpResponseBase _response;
public ReportingServiceProxy()
: this(new HttpResponseWrapper(HttpContext.Current.Response))
{}
public ReportingServiceProxy(HttpResponseBase httpResponse)
{
_response = httpResponse;
//...
}
public FileContentResult GetReport(string reportName, ReportFormat reportFormat, IList parameters)
{
// ...
_response.AddHeader("cache-control", "must-revalidate");
}
}
public class HttpResponseFake : HttpResponseBase
{
public override void AddHeader(string name, string value)
{
Console.WriteLine("Header {0} added with value: {1}", name, value);
}
}
[Test]
public void should_get_report_from_test_env()
{
var reportingServiceProxy = new ReportingServiceProxy(new HttpResponseFake());
...
}
MPMoviePlayerController
I need implement the following two features in most of my iOS apps, e.g., iClip:
- auto-play-next-episode if available
- auto resume from the stop point last time stopped/pasued
By looking into the developers doc, the first one is easy to do, just observe the MPMoviePlayerPlaybackDidFinishNotification:
-(void)playMovieAtURL:(NSURL*)theURL
{
MPMoviePlayerViewController* mpvc =[[MPMoviePlayerViewController alloc] initWithContentURL:theURL];
[self presentModalViewController:mpvc animated:NO];
double stoppedPoint = [[NSUserDefaults standardUserDefaults] doubleForKey:[mpvc.moviePlayer.contentURL absoluteString]];
if (stoppedPoint > 0.0) {
mpvc.moviePlayer.initialPlaybackTime = stoppedPoint;
}
[[NSNotificationCenter defaultCenter] addObserver:self
selector:@selector(myMovieFinishedCallback:)
name:MPMoviePlayerPlaybackDidFinishNotification
object:mpvc.moviePlayer];
if (![self isOS5])
{
mpvc.moviePlayer.useApplicationAudioSession = NO;
}
self.mpvc = mpvc; //retain it
[mpvc release];
return;
}
// When the movie is done,release the controller.
-(void)myMovieFinishedCallback:(NSNotification*)aNotification
{
if ([[aNotification object] isKindOfClass:[MPMoviePlayerController class]])
{
MPMoviePlayerController* theMovie=[aNotification object];
[[NSNotificationCenter defaultCenter] removeObserver:self
name:MPMoviePlayerPlaybackDidFinishNotification
object:theMovie];
[self dismissModalViewControllerAnimated:NO];
theMovie = nil;
[theMovie release];
}
NSDictionary *userInfo = [aNotification userInfo];
int reason = [[userInfo objectForKey:@"MPMoviePlayerPlaybackDidFinishReasonUserInfoKey"] intValue];
NSLog(@"clip finished because %d. play next one if available.", reason);
// ios 4 always return MPMovieFinishReasonPlaybackEnded even it's user exit it, a bug. don't provide play next feature for ios4 users then.
if ([self isOS5] && reason == MPMovieFinishReasonPlaybackEnded)
{
[self playNextEpisodeIfAvailable];
}
}
Notice the finished reason in iOS 4 (bug) is always MPMovieFinishReasonPlaybackEnded, I had to implement the same feature in MPMoviePlayerPlaybackStateDidChangeNotification for iOS 4 users:
-(void)myMovieStateChangedCallback:(NSNotification*)aNotification
{
if ([[aNotification object] isKindOfClass:[MPMoviePlayerController class]])
{
MPMoviePlayerController* theMovie=[aNotification object];
if (theMovie.playbackState == MPMoviePlaybackStateStopped && theMovie.currentPlaybackTime == theMovie.playableDuration) {
NSLog(@"clip finished!");
if ([self isOS5]) {
}else{
[self playNextEpisodeIfAvailable];
}
}
For second feature, saving stop point when movie exit, I also use the MPMoviePlayerPlaybackStateDidChangeNotification to catch the currentPlaybackTime, I’ve tried catch it in MPMoviePlayerPlaybackDidFinishNotification, the value is zero, too late.
WIF, ADFS, OIF, OpenSSO/OpenAM
WIF provide a way to separate security concern out from dotnet applications. Outsource identity/authorization task to a 3rd party IDP/STS really loose the structure of classic web apps, no need to write those user registration / profile management / group assignment common task again and again, and the app is only loose coupled with IDP through Claims.
The idea is neat, but finding a right STS is not easy.
ADFS 2.0 can handle LDAP user without any problem, the off-shelf features don’t cover extranet users.
The out-of-box WIF template provides a local STS for dev purpose only, and it’s using WS-Fe
WIF application is only talking to STS through WS-Federation protocol, (SAML 2.0 Protocol support can be downloaded here, I haven’t try it yet), while Oracle STS speaks WS-Trust, Web Service only?
Right now the solution is use ADFS as gateway between WIF app and 3rd-parth IDP.

Some difficulties we encountered:
Not getting home-realm discovery page from ADFS
This problem was caused by when importing claim provider trust federation metadata into ADFS, the endpoint was not imported, ADFS only accept https endpoint.
HTTP Error 503 on adfs/services/trust and get the service unavailable after STS login successfully
This happens to StarterSTS and CustomSTS, for StarterSTS, folloew this post to add relying party key into StarterSTS and export token decryption certificate. Note the original post missed a key name in configure demo code.
For CustomSTS (WIF test STS), the ADFS replyToAddress needs to set manually based on the Federation Service identifier, an example can be found here as well.
public class CustomSecurityTokenService : SecurityTokenService
{
protected override Scope GetScope( IClaimsPrincipal principal, RequestSecurityToken request )
{...
foreach (var key in ConfigurationManager.AppSettings.AllKeys)
{
_logger.Debug(string.Format("checking [{0}] with [{1}]", scope.AppliesToAddress, key));
if (string.Equals(scope.AppliesToAddress.ToLower(), key.ToLower()))
scope.ReplyToAddress = ConfigurationManager.AppSettings[key];
}
<appSettings>
<add key="http://adfsserver/adfs/services/trust"
value="https://adfsserver/adfs/ls/"/>
</appSettings>
SP doen’st support urn:oasis:names:tc:SAML:1.1:nameid-format:unspecified
This happens to OpenSSO, we finally used SAML Tracker addon in Firefox to figure out it’s in SAML token ADFS sending to IDP. Then followed this post to changed the default format from unspecified to something else.

iBoozeHunter
iClip
iGameTimer2
iGameTimer Lite
maodou