Archives for October, 2010
Few months ago Miguel de Icaza posted on Twitter message about explicitly stating your member private access modifier (or anything that’s default) and that he doesn’t understand it, the he wants his code be succinct.
class Foo
{
private int _bar;
// or?
int _bar;
}
I, as a lot of other people, replied, that I want to be explicit about my code. Tell exactly what I want. However I had a sneaking idea in my brain since this message – because I like my code, my thoughts in code, to be succinct – and I decided give it a try. I removed all the explicit access modifiers that are actually default from ID3 renamer‘s code base a start. Well, after working with it some time, I didn’t noticed anything confusing. There’s only couple of default rules, so it’s not like you would need to learn a lot of new stuff. And, for me, because there’s less text in source code, it looks more readable for me, especially if I’m just checking something out using notepad-like (no highlighting) viewer.
I’m now following this “rule” for every new code I’m writing. I like it when I spin my head around something that looks completely useless to think about, for a first sight, but later shows up to be a good idea or true.
Chcete-li se dozvědět zajímavé informace o Sync Frameworku – nástroji pro (nejen) snadnou synchronizaci souborů, databází, … – můžete zajít na mou přednášku na WUGu [akce.altairis.cz] 26.10.2010 (úterý).
Recently I was running NUnit tests we’re using for .NET provider and I got an idea to compare different Firebird versions. Not because it’s any standard test (frankly it’s way not), but because I have to run the tests against different Firebird versions anyway and I see the time. It’s not about the time itself, rather about the performance improvement (or not) between two versions.
I used NUnit’s GUI to run tests. Tests are run in one thread and I was trying to make the environment same as much as possible. The server was processing nothing else, only the commands from tests. The commands in tests are nothing special, it’s more about testing the provider’s correctness than stressing server. But some processing is there. More processing is done in set up (and tear down) methods where database, objects are created and tables are filled with some data. A lot of commands, but simple ones. With this configuration you can easily see, that cache isn’t useful here as well as improved SMP support in 2.5.
Both versions we’re current latest from 2.1 (2.1.3 SuperServer) and 2.5 (2.5.0 SuperClassic) trunks, official builds, 32-bit, default configurations.
I can clearly state, that 2.5 is faster. About 13%. Is it a lot of not? Who cares. This test was very far away from real world scenario, hence for you application it might be even better (or not 8-)). Good news is, that 2.5 really is faster, no empty promises.
I was recently thinking about phones and how these, now so called smart, device work together with our life. And I have to say my conclusions are not good. 
I have a feeling, that the smartphone are still with us as another device in pocket. Isolated, kind of, from what’s going on around. I would like to see integration, more tight integration. But not limiting.
Take, for instance, how the phone behaves according to different ring profiles in various places. I can imagine it being after some time able to recognize where I’m. No moves, 10p.m.-some alarm and still repeating wifi network name connected to? Well that’s possible my home (notice I’m not taking into account well offering GPS or least last position before you got into building) and I may probably always want to set up same ring profile when I get into these “conditions” anytime from 3p.m. (just example). And not ask me what ring profile to use. Observe profiles being set for a while and offer me one later. Chances are good you’ll be right. The phone isn’t doing some environment changing decisions on behalf of you. It is doing smart observations and offering you ways to make your life easier.
I can provide dozens of similar “guesses”. But to take it to next level the devices should do two more things. Learn what’s working and what isn’t. Are all input variables based on time in 90% wrong? Well you’re (example) maybe traveling a lot and hence having random patterns there. The device could, instead of mentioned time, take into account calendar, for instance.
That’s learning or intelligence. We may consider being part of algorithm as well. But with next point, it’s really next level. And it’s inter-device communication. One could say incorporating cloud, a word so often used today. But why not. Devices would recognize (and ask for confirmation, if needed) “friend” devices. Like wife’s and husband’s phone. But not only these obvious cases. Let me show you example. You’re at airport and the flight is delayed, you start hunting for wifi to do some work, but you’re low on battery (read further to know why the phone thinks you’ll not do it on energy left in battery). But another people found power outlet somewhere near your gate and are charging their phones. Wouldn’t it be nice if the phone would offer you this information because it knows you are going to need charging because according to the location you’re five hours from your final destination stored in appointment in calendar (that’s why are you low on battery)? And the devices would not share only “there’s power outlet”, but “I was low on battery and found power outlet near me, it might be worth offer your owner to recharge you if you’re in similar conditions, are you aware of it?”. Know what I mean?
Sharing information is just part of the solution. Learning, passing ideas and decisions and algorithms, that’s next step in experience, in life integration, in life helping. Sadly we’re not there yet.
I recently discovered nice feature of NHibernate. It’s called MultiQuery (but the name doesn’t matter). The idea behind is simple. Instead of sending multiple queries one by one and melting performance of your application in network latency, send all in one batch.
I read couple of articles about it. Later something in my head started to working and I had an idea about trying to do it in Entity Framework. I had a basic concept in my head in couple of minutes and I told myself I’ll try to do it, but I’ll not invest too much time into it. Just quick’n'dirty brain exercise for Saturday (alike Bart de Smet‘s Crazy Sundays).
The concept was simple. Record couple of ObjectQuery objects, get commands out of these, create one huge batch, re-wire parameters (more about that later) and get results.
public class MultiQuery
{
struct QueryRecord
{
public ObjectQuery Query { get; set; }
public Type Type { get; set; }
public static QueryRecord Create<T>(ObjectQuery<T> query)
{
return new QueryRecord() { Query = query, Type = typeof(T) };
}
}
#region Fields
ObjectContext _context;
List<QueryRecord> _queries;
#endregion
#region Constructors
public MultiQuery(ObjectContext context)
{
_queries = new List<QueryRecord>();
_context = context;
}
#endregion
#region Public Methods
public MultiQuery Add<T>(ObjectQuery<T> query)
{
if (query == null)
throw new ArgumentNullException("query");
_queries.Add(QueryRecord.Create(query));
return this;
}
public MultiQuery Add<T>(IQueryable<T> query)
{
return this.Add(query as ObjectQuery<T>);
}
public IEnumerable<ObjectResult> Execute()
{
IDbConnection storeConnection = ((EntityConnection)_context.Connection).StoreConnection;
using (IDbCommand cmd = storeConnection.CreateCommand())
{
IDataParameterCollection parameters = cmd.Parameters;
cmd.CommandText = CreateCommand(_queries.Select(q => q.Query), cmd.CreateParameter, ref parameters);
bool shouldClose = (_context.Connection.State == ConnectionState.Closed);
try
{
storeConnection.Open();
using (IDataReader reader = cmd.ExecuteReader())
{
int cnt = 0;
do
{
yield return _context.Translate(_queries[cnt].Type, reader);
cnt++;
} while (reader.NextResult());
}
}
finally
{
if (shouldClose)
storeConnection.Close();
}
}
}
#endregion
#region Private Methods
string CreateCommand(IEnumerable<ObjectQuery> queries, Func<IDataParameter> parameterCreator, ref IDataParameterCollection parameters)
{
List<string> commands = new List<string>();
int cnt = 0;
foreach (var q in _queries.Select(q => q.Query))
{
string query = q.ToTraceString();
foreach (var p in q.Parameters)
{
IDataParameter parameter = parameterCreator();
parameter.ParameterName = string.Format("@p{0}", cnt++);
parameter.Value = p.Value;
parameters.Add(parameter);
// Not good. Better (and still easy) idea?
query = query.Replace(string.Format("@{0}", p.Name), parameter.ParameterName);
}
commands.Add(query);
}
return string.Join(";" + Environment.NewLine, commands);
}
#endregion
}
static class MultiQueryExt
{
internal static ObjectResult Translate(this ObjectContext context, Type type, IDataReader reader)
{
// ObjectResult<TElement> Translate<TElement>(DbDataReader reader)
object result =
context
.GetType()
.GetMethod("Translate", new[] { typeof(DbDataReader) })
.MakeGenericMethod(type)
.Invoke(context, new object[] { reader });
return (ObjectResult)result;
}
}
I’m here fully utilizing new Translate method in Entity Framework 4 (for v1 similar method is available in EFExtensions). The rest is done using pure ADO.NET. It’s worth noting, that this code, same as in NHibernate, works only if the database and the underlying provider supports processing more queries in one command (i.e. Microsoft SQL Server does, but Firebird does not).
Also small notice to parameters. I’m doing simple replace and that’s dumb. It may fail and produce wrong results, but in very rare cases. So you should test thoroughly. The case when it produces wrong results is, when you write query in where you use directly (not as a variable etc.) string that is same as parameter name (i.e. p__linq__<number> for SqlClient or p<number> for FirebirdClient). As you are in control of these strings you can change the code to use a variable, for instance.
A lot of “fetching” methods in Entity Framework supports also MergeOption. Adding overload for Execute I’m leaving as exercise for readers. Likewise for the Entity SQL queries.
More than a year ago I wrote about tab closing button in Visual Studio 2010 making me crazy. I still don’t know whether the new placement is better or not. But it’s same as in the most common tabbed applications – browsers. Probably in some scenarios it’s better to have it in fixed place in upper right corner and other scenarios fit better for placing it to the tab.
Anyway I’m using Visual Studio 2010 as my daily environment for almost a year and I got used to it. The problem now is exact opposite. Working in Visual Studio 2008 I’m instinctively moving my mouse near to the tab I’m going to close instead to the upper right corner.
As the Visual Studio will move forward, we’ll probably forget about the old placement. Of course up until some UX designer will move the button to different place. 
If you’re composing queries in LINQ on various places, utilizing the delayed execution, you might be surprised, that some of your dynamically added OrderBys are not in final query.
Imagine we have a simple table with a and b columns (and primary key). If you write following query with ordering adding (it can be in different method etc.) …
IQueryable<OrderingTest> tmp1 = context.OrderingTest;
tmp1 = tmp1.OrderBy(x => x.a);
tmp1 = tmp1.OrderBy(x => x.b);
//Console.WriteLine((tmp1 as ObjectQuery).ToTraceString());
… the result will contain sorting based only on b column. That’s because the last OrderBy took the precedence.
To make it work as expected you have to write it like this.
IQueryable<OrderingTest> tmp2 = context.OrderingTest;
tmp2 = tmp2.OrderBy(x => x.a);
if (tmp2 is IOrderedQueryable<OrderingTest>)
tmp2 = (tmp2 as IOrderedQueryable<OrderingTest>).ThenBy(x => x.b);
//Console.WriteLine((tmp2 as ObjectQuery).ToTraceString());
The secondary (and further ordering) needs to be done via ThenBy method, which is available on IOrderedQueryable<T>. That’s the reason for casting.
Sure it’s always safer to do this in one place directly, but sometimes the query is and has to be build on various places. Then think if a call to OrderBy could or couldn’t be at some place before.
You can always do some “dummy sort”, like _ => 0, initially and then use only ThenBy, but I personally don’t like playing with the intelligence of optimizer. It may have bad impact on performance.
Po delší době, kdy mě vytočilo používání mně a mě špatně, tu mám další kousek. Normálně mírné excesy v používání slova permanentka přecházím, ale tento týden jsem “pernamentka” slyšel tolikrát, že si musím ulevit.
Takže. Slovo “pernamentka” je blbost. Nic takového není. Správně je permanentka (viz Ústav pro jazyk český). Na výše uvedeném odkazu je i chybně použivaný tvar uveden.
Závěr je jednoduchý. Naučte se slovo správně, ať nevypadáte jako blbci (a ať mám já klidné spaní ).
Common Table Expressions are a tool to express things in SQL without need to write long queries or use procedural SQL. The ability to write recursive queries is even nicer and yesterday I had an idea how to use it little bit differently.
Often I’m generating data for reports and I’m trying to do as much as I can on database server. One common problem I’m facing with these reports is the need to include some “zero” data even for, in this case, days, where no data were available.
Imagine, you’re doing something like this (Firebird syntax).
select data.d, sum(data.i) from
(
select current_date as d, 1 as i from rdb$database
union all
select current_date, 1 from rdb$database
union all
select current_date+1, 2 from rdb$database
union all
select current_date+3, 5 from rdb$database
union all
select current_date+3, 2 from rdb$database
) data
group by data.d
It’s OK, but you’ll get data only for date that are present in database. But because you’re generating report for whole month, as I did, it would be nice to have there the data too even with “zeros”. Because then the logic on application side is much simpler, you just need to deal how to present the report’s data.
If you have CTEs, it’s pretty easy to use the recursiveness and generate the sequence, join it with original data and do what you have to do.
select range.d, coalesce(sum(data.i), 0) from
(
select current_date as d, 1 as i from rdb$database
union all
select current_date, 1 from rdb$database
union all
select current_date+1, 2 from rdb$database
union all
select current_date+3, 5 from rdb$database
union all
select current_date+3, 2 from rdb$database
) data
right outer join
(
with recursive padding as
(
select current_date as d from rdb$database
union all
select dateadd(day, 1, d) from padding where d < dateadd(month, 1, current_date)
)
select * from padding
) range
on (data.d = range.d)
group by range.d
I'm using here right outer join just to keep the original data above the sequence I'm generating. But you can swap the tables and use left outer join as it may be easier to read and maintain.
The code is pretty straightforward and if you used CTEs ever before, you'll be able to read. I'm simply generating the dates until I'm ready with the sequence (in this case one month) and then joining with original data. The outer join will ensure the holes are filled and nulls are there. These nulls are later processed with coalesce to "zero" values.
If you want to directly control the number of elements in sequence, simply use some counter.
with recursive padding as
(
select current_date as d, 0 as cnt from rdb$database
union all
select dateadd(day, 1, d), cnt+1 from padding where cnt < 10
)
select * from padding
Previous two functions (function 1, function 2) I presented were doing something that wasn’t core part of LINQ and it was up to you to create it. On the other hand, this function is different. It’s just a helper to simplify writing of left outer join (or right outer join, depending on what collection you consider to be on left/right side). Not because it’s hard to write it, but because it involves couple of lines and repeating it all the time is just boring. 
internal static IEnumerable<TResult> LeftOuterJoin<TOuter, TLeft, TKey, TResult>(this IEnumerable<TOuter> outer, IEnumerable<TLeft> left, Func<TOuter, TKey> outerKeySelector, Func<TLeft, TKey> leftKeySelector, Func<TOuter, TLeft, TResult> resultSelector)
{
return
from o in outer
join r in left on outerKeySelector(o) equals leftKeySelector(r) into j
from r in j.DefaultIfEmpty()
select resultSelector(o, r);
}
Nothing tricky. You can find this in many examples, I just wrapped it into method and parametrized it a little. Enjoy.
Similarly to previous method for Interleaving two IEnumerables I needed one method for doing kind of aggregate, but also to know about the intermediate steps. I could abuse the Aggregate method either directly or indirectly but I’m always more happy with clean solution.
The method is pretty simple (but before I did some rework, it was ugly ). It takes, apart from the IEnumerable, two functions. One for setting up the initial value for the first item and one for getting result for next step.
internal static IEnumerable<TResult> IncrementalAggregate<TSource, TResult>(this IEnumerable<TSource> data,
Func<TSource, TResult> init,
Func<TSource, TResult, TResult> nextResult)
{
bool first = true;
TResult intermediate = default(TResult);
foreach (var item in data)
{
if (first)
{
intermediate = init(item);
first = false;
}
else
{
intermediate = nextResult(item, intermediate);
}
yield return intermediate;
}
}
With it you can do i.e. summing the numbers and know what the intermediate sums were. Yes, sounds weird, but you might need it, one day as I did.
I was recently doing some work related to custom reporting and I needed to simply interleave two streams of data (it was actually same source, but different items selected). Kind of select first item from the first stream, first from the other, second from first, second from other etc. No big deal.
To make this easily doable I created simple extension method for me.
internal static IEnumerable<T> Interleave<T>(this IEnumerable<T> first, IEnumerable<T> second)
{
using (IEnumerator<T>
enumerator1 = first.GetEnumerator(),
enumerator2 = second.GetEnumerator())
{
while (enumerator1.MoveNext())
{
yield return enumerator1.Current;
if (enumerator2.MoveNext())
yield return enumerator2.Current;
}
}
}
It keeps reading elements from the first stream and if there’s enough in the second stream then interleave. If the second one isn’t “long” enough, it’ll keep returning only items from first one. If the second one is “longer”, it’ll stop when the first one is empty. If you need handle these cases differently, you can either change the method or preprocess the streams before using this method.
Today the Firebird 2.5 is being released. You can read the press release and most importantly release notes. And sure, you can download it and use/test/deploy.
I’m not going to repeat what’s written in documents above. Instead I’m going to provide some of my information about using it and some thoughts from .NET provider view.
To be honest I’m using Firebird 2.5 in production environment since betas (I made a small mistake and wasn’t able to easily go back and was lazy to do the long process ). And I’m more than pleased with the stability. Even the betas were more stable than 2.1, where I, probably thanks to environment, experienced I/O problems from time to time. The language features are nice too. I don’t have any personal favorite, all are great addition to whole ecosystem. Actually I do have. It’s not a language feature, but a fixed behavior. And it’s affecting .NET provider.
In some cases the left outer join produced wrong results. Unluckily this error was exposed when discovering database structure for Entity Framework, so even you could use 2.1 in production, for development (at least for model generation) you had to use prerelease versions of 2.5 and that’s, if nothing else, little bit inconvenient. Not taking into account some internal protocol and core improvements that are now exposed in provider as well. I.e. cancellation of running command to pin point one.
I could write more and more about the .NET provider and new version. True to be told, I like every improvement I do in provider that may help other to fully unleash the power of Firebird.
Congratulation to us, the Firebird Project, especially the core team. And also to you, users, I hope you’ll enjoy and like the new Firebird 2.5 version as we (I) do.
Note: The MindTheBird campaign team will run a webinar today at 13:00 GMT in anticipation of the launch of Firebird 2.5 Final Release. See the details.
