tag:blogger.com,1999:blog-59301292024-03-08T09:35:53.337-04:00Aquarium BlogLife, software, politics, arts, and naive future prediction (This blog is in suspended animation)Diegohttp://www.blogger.com/profile/06211524340372500555noreply@blogger.comBlogger318125tag:blogger.com,1999:blog-5930129.post-8087159950468180702007-11-09T19:40:00.000-04:002007-11-09T19:43:35.445-04:00Moving to MSDNI haven't decided yet, but it is very likely that I will stop blogging here for some time.<br /><br />For some background, I have moved to the sate of Washington and now I am working for Microsoft.<br /><br />I have my <a href="http://blogs.msdn.com/diego">new blog</a> setup on MSDN. I was glad to find that no other Diego was blogging at the company :)Diegohttp://www.blogger.com/profile/06211524340372500555noreply@blogger.com1tag:blogger.com,1999:blog-5930129.post-31110316866167902332007-09-10T22:31:00.001-04:002007-09-11T08:29:45.101-04:00So much to learn!<p>I only followed halfway of the rabbit hole about the idea of "composable data (entity) services" and found that much has been written and debated about the topic. It seams, for instance, that part of <a href="http://blogs.msdn.com/nickmalik/archive/2007/05/24/is-an-entity-service-an-antipattern.aspx#comments">the</a> <a href="http://soa-eda.blogspot.com/2007/05/two-worlds-of-soa.html">SOA</a> <a href="http://udidahan.weblogs.us/2007/06/08/entity-services-rollup/">intelligentsia</a> has been discussing if the concept of entity services could be some kind of anti-pattern. </p> <p>On the other side, I think Astoria's value proposition is very solid, at least for the scenarios currently targeted (which I think are more oriented to <a href="http://en.wikipedia.org/wiki/Mashup_%28web_application_hybrid%29">mashups</a> that do all data aggregation on the client side). Also, <a href="http://blogs.msdn.com/pablo">Pablo Castro</a> addressed some of the concerns that could apply to Astoria shortly after MIX07 on his own blog.</p> <p>The Entity Framework is also such a distinct beast, that could shift the balance on what is a good practice (i.e., by making entity services very easy and inexpensive to own).</p> <p>I would really like to get the whole picture, but this will have to wait for now... </p> Diegohttp://www.blogger.com/profile/06211524340372500555noreply@blogger.com0tag:blogger.com,1999:blog-5930129.post-14973751501449776122007-09-09T08:21:00.001-04:002007-09-09T21:46:00.528-04:00Entity Coupling Service another name for Composable Data Service<p>Mats Helander apparently explores the same subject of "composable data services" in <a href="http://www.matshelander.com/wordpress/?p=71">his post</a>, although he seems to arrive from a different venue as <a href="http://www.base4.net/Blog.aspx?ID=566">Alex</a>. </p> <p>For me it smells more and more like <a href="http://blogs.msdn.com/astoriateam/">Astoria</a> + "Aggregation" (not that Astoria doesn't plan to support composition or aggregation, this I am not sure of).</p> <p>I noticed that in some comments to that post, <a href="http://udidahan.weblogs.us/">Udi</a> expresses his opposition. I am not sure he opposes the entity services Mats refer to, or the idea of aggregating multiple data sources. I wish he explained his stance in more detail. Mats explanation of Entity services makes me believe the refers to services that only handle a single kind of entity each, which to my intuition sounds like exaggeratedly granular.</p> Diegohttp://www.blogger.com/profile/06211524340372500555noreply@blogger.com3tag:blogger.com,1999:blog-5930129.post-67099239673170645392007-09-04T17:25:00.000-04:002007-09-04T17:29:39.700-04:00Updated visited countriesJust needed a little distraction and found someone visited my <a href="http://diegov.blogspot.com/2004/04/my-visited-countries-from-world66.html">old post</a>:<br /><br /><br /><img src="http://www.world66.com/myworld66/visitedCountries/worldmap?visited=CAUSBBUVDMDOHNMQANPRVIARBRCLCOECPEUYFRDEGRITNLESVATR" width="420" /><br /><br /><a href="http://www.world66.com/myworld66">create your own visited country map</a>Diegohttp://www.blogger.com/profile/06211524340372500555noreply@blogger.com0tag:blogger.com,1999:blog-5930129.post-34995870194668069702007-09-04T09:53:00.001-04:002007-09-11T10:08:19.749-04:00Note to self: Always use Windows Live Writer<p>I admit that I do much editing of my posts post-publishing. This is specially true for a post like the last one, in which I try to explain a fairly complex idea with my rudimentary English.</p> <p>But I usually need to look at the finished post on the actual blog layout to detect most errors and readability problems. So, the process usually goes like this: I open my blog in the browser and start reading. When I find something I want to change, I open the <a href="http://www.blogger.com/">Blogger</a> page in other window and start correcting it. Unfortunately, Blogger's editing pane won't grow enough to give me a good view of the text I am editing. Another annoyance is that the spell checking won't always work because of pop-up blockers. </p> <p>I had tried <a href="http://writer.live.com/">Windows Live Writer</a> before and I was somewhat impressed, but it was today that I really began to appreciate the difference. </p> <p>What I like the most about it:</p> <ol> <li>WYSIWYG, full-screen editing. <li>Managing multiple blogs (it supports most blogging platforms). <li>Integrated spell checker. <li>Paste special/Thinned HTML.</li></ol> <p>There is no reason I will use Blogger's user interface again to post a new entry.</p> Diegohttp://www.blogger.com/profile/06211524340372500555noreply@blogger.com0tag:blogger.com,1999:blog-5930129.post-70845608834834758282007-09-03T13:21:00.000-04:002007-09-05T23:36:15.012-04:00Around the globe with composable data services<p><a href="http://www.ayende.com/">Ayende</a> and <a href="http://www.base4.net/Blog.aspx">Alex</a> have been having an interesting conversation on the subject of data access layer componentization, in the light of some new features that are appearing in Microsoft's Entity Framework and some previous work by Alex on <a href="http://www.base4.net/">Base4.NET</a>.<br><br>You can find some of the most relevant posts of the conversation <a href="http://www.base4.net/blog.aspx?ID=559">here</a>, <a href="http://www.ayende.com/Blog/archive/2007/09/01/Data-Layer-ComponentizationAgain.aspx">here</a> and <a href="http://www.base4.net/Blog.aspx?ID=566">here</a>.<br><br>I read Ayende's answer last night and Alex answer this morning. I was about to write a comment, but it suddenly grew too much. Alex lives in New Zealand and Ayende, I think, lives in Israel. I am writing this at almost 3:00 PM (GMT-4, Caribbean time). I hope they are sleeping right now, so I will have time to do the usual editing after publishing!<br><br>I agree with Alex that most of Ayende's concerns could be addressed by the composable EFx data services Alex envisions (he actually prefers to use the term "dataservers", but I think it is opportune to borrow some jargon from <a href="http://blogs.msdn.com/pablo/archive/2007/08/09/a-data-service-is-not-a-database.aspx">Astoria</a>).<br><br>Note: In this case we have used the terms "composable" and "componentization" in the sense that the service can aggregate information from multiple backends under a single conceptual model. Maybe we should find a more explicit term to avoid overlaps with the use of "composability" elsewhere.<br><br>Looking at Ayende's diagram I agree he did not get the complete picture Alex was painting. To his favor, one must admit that the composable data services Alex talks about are still not even “vaporware”.<br><br>I really just want to add two elements to the conversation:<br><br>First, if you added caching of read-only data as a feature of the data service, you would get a better substitute for the ETL process that Ayende mentions (Note To Alex: You can consider this a feature request!).<br><br>Second, while Ayende’s preferred solution may look very good and the simplest thing to do in some scenarios, IMHO its main weakness is that it does not scale. Let me try to explain it with an extreme example:<br><br><em>Suppose an enterprise has 5 mayor systems serving 5 departments, each of them with its own data silo. One day, each department contracts a consultant to help them do some data integration with the other systems (not that this should ever happen in real life!).<br></em><br><em>A few weeks later, each consultant comes up with a solution very much like the one Ayende explains: Each one contains its own schema for the data coming from the other 4 databases, each of these new schemas is fed by a separate ETL process, etc.<br></em><br>Now that the five consultants took their money, let’s analyze what the customer actually got:<br><br>ONE TIME COST: Contained in each of the original 5 systems there is a subset of the data that needs to be shared. But instead of sharing it, the consultants decided simultaneously that the easiest path for each of them was to duplicate this data. So, in the end, for this subset of data that needs to be shared, the increase of storage is up to 5 x 4 = 20 fold! This will not only cost hardware: The schemas for this subset have been reinvented up to 20 times too and 5 different ETL processes had to been designed, implemented and tested.<br><br>RELIABILITY: For simplicity sake, we will only consider uptime, which measures the ability of the system not to "go down", and not its ability to maintain data consistency. If you do the math, I think you will see that in theory the customer's infrastructure is now more tolerant to failure. However, in "reality" the infrastructure is now much more complex, and hence much of this advantage is "lost to entropy" (every time something goes wrong, fixing it is more complex). You could have instead invested the same money in redundancy for each of the 5 original systems. While two-fold redundancy buys less reliability than five-fold redundancy, most failover solutions won’t add so much complexity.<br><br>MAINTAINABILITY: I don’t understand quite well Ayende’s points regarding maintainability, because on the event of a single schema modification, he still needs to at least revise his ETL code. Although the system could surely be kept running for hours on outdated data (improving uptime, not maintainability), eventually he would need to adjust it. In my extreme example, any single schema change can potentially affect all 5 systems! In contrast, if you could create a single compound EFx data service, you would probably just compensate for the changes by adjusting the mapping, and only once. <strong>UPDATE:</strong> I see I was assuming here a "static" definition of maintainability, that is completely orthogonal with uptime. I may reconsider this argument, but it doesn't affect the main point.<br><br>SECURITY: I don’t clearly see Ayende’s point regarding security either. I think you need some means to perform authentication and flexible authorization, and to protect critical data, no matter if you are exposing it as a data service or if you make it available to an ETL process and then to users. Anyway, we still don't know exactly what shape will security take in EFx and Astoria.<br><br>PERFORMANCE: How the new system will actually perform is impossible to predict (due to too many factors that are not detailed in the example). However, we can easily observe lots of overhead in moving the same data among several servers. Once you have 5 copies of the data, you will probably see some performance improvement because of locality and parallelism. But the same effect could be achieved in a data service by using caching and conventional scale out measures. In such a case, schemas would not be unnecessarily complicated and consistency would be easier to maintain.<br><br>My point is that this data duplication approach, while simple at first, is a path that an organization should not take many times. Once you have, say, three of these processes in operation, it will probably be too much pain to add another one.<br><br>This is only how things happens in a fictitious example. And Oren only talked about one system doing this. However, my thesis is that this scenario is not too much detached from how things would go in real life. </p> <p>I think the consultants would probably not talk much to each other, and they would probably never come up with an integrated solution. Why?<br><br>1. Business reasons: Simply put, each consultant is set to do what is best for his project and revenue in the short term, not what is good for their customer in the long term. They will optimize locally, not globally.<br><br>2. Most important of all, a technical reason: Unfortunately, there is currently no simple way of accomplishing the integration that the consultants could agree upon. This is precisely the need that composable EFx data services could address.<br><br>To satisfy the data integration needs of a company like the one in the example, a new kind of data access technology is needed: One that allows you to easily build data services that are composable, that can extract data from virtually any source, that expose a very high level (conceptual) data interface, which support flexible mapping, and that everyone can talk to using standard protocols.<br><br>I think that Alex and I agree that most pieces of this solution are already beginning to appear. </p> <p>The last paragraphs sound a lot like marketing :D But seriously, if the Data Programmability Team were going to be built such a thing, it would be yet another reason for me to be excited.</p>Diegohttp://www.blogger.com/profile/06211524340372500555noreply@blogger.com1tag:blogger.com,1999:blog-5930129.post-40608322400763156692007-06-27T02:09:00.001-04:002007-07-14T16:07:24.780-04:00Delphi Roadmap<p>I found today the <a href="http://dn.codegear.com/article/36620">Product Roadmap for Delphi</a>, through <a href="http://community.devexpress.com/blogs/ctodx/archive/2007/06/19/fahrenheit-vcl.aspx">a post in Julian Bucknall’s blog</a> (Julian is the CTO at <a href="http://www.devexpress.com/">DevExpress</a>) . <p>There seem to be some good news, but it still feels like everyone is avoiding the sad truth: The declining relevance of Delphi in the market. <p>I happen to be a .NET developer that holds some remote but very nice memories of coding on Object Oriented Turbo Pascal and Delphi. <p>I respect Delphi. I appreciate the importance of the existing codebase and the skill set of Delphi developers. I admire the people that worked on its design and the people that are working on it now. Like so, I believe in Delphi as a language and in Delphi as an “ecosystem”. I want those things to remain relevant. Actually, I think those are the core assets <a href="http://www.codegear.com/">CodeGear</a> still holds. <p>I don't really know if it is possible to build a sustainable business model solely on those essential values, but once you have this, next step would be to listen to what really make sense for developers. <p>My personal take: I believe in managed code and I could not care less about Win32. On one side, I see <a href="http://sessions.visitmix.com/">all sorts of cool things</a> happening around the CLR. On the other side, the latest incarnation of the main Win32 vehicle, that is, Windows Vista, now comes with .NET 3.0 installed. <p>You may not completely love Vista, but no doubt that in a couple of years, most Windows computers out there will have at least .NET 3.x installed. Win32 is just a necessary evil, and perhaps it is not even so necessary! <p>The thing I love the most about .NET is the amount of existing code I can use and extend, regardless of the language it was originally written in. I also like the way it works with Unicode from the beginning. I love the way it helps me move to 64bits almost seamlessly. These things are mentioned in the Roadmap, and Julian mentions them as big issues (breaking changes for existing code). <p>I believe in the value of Visual Studio. I like using the designers: Windows Forms and WPF, the new Web Designer, WF, DSL, Team System, and the bunch of new things that will come in VS 2008, just like the new features in C# and VB. <p>Visual Studio may be not perfect but there are plenty of excellent third party extensions filling the holes. I would love to be able to use things like <a href="http://www.devexpress.com/Products/NET/IDETools/Refactor/">Refactor Pro!</a> or <a href="http://www.jetbrains.com/resharper">ReSharper</a> with Delphi. <p>Even further, it would be sweet to be able to use Delphi with NAnt, MbUnit, NDepends, Windsor, TDD.NET and the whole ALT.NET stack. <p>It would be awesome to run Delphi on Linux via Mono. <p>It would be fantastic to run Delphi code on a Mac via Silverlight. <p>It would be great to run ASP.NET AJAX applications on any browser, powered by Delphi code on the back end. <p>So, this is my wish list: <p>0. I said once I wanted Microsoft to buy Delphi from Borland. I think cannot count on this anymore, but what the hell... <p>1. I would like to see CodeGear to focus on Delphi the language, basically making it really fly on .NET 3.5 as soon as possible, complete with generics and LINQ. <p>2. I would like to see someone (CodeGear or else) write Visual Studio bindings for Delphi, and shipping Delphi on the Visual Studio 2008 Shell. <p>3. I would like to see someone (CodeGear or else) to take charge of a good compatibility/migration story for VCL. </p><p>4. I am cool with CodeGear wanting to continue the development of Win32 Delphi, to keep their C++ and IDE business, and to make a new Ruby on Rails IDE. I guess they, better than I, can assess if they are contributing anything really new and significant in those areas. </p>Diegohttp://www.blogger.com/profile/06211524340372500555noreply@blogger.com0tag:blogger.com,1999:blog-5930129.post-5483493412210576632007-05-30T00:19:00.000-04:002007-09-07T08:09:21.998-04:00IsNullOrEmpty for IEnumerable<p>This is not extremely relevant, but String.IsNullOrEmpty() has become a very popular time saver, and I think the same concept should be applicable to arrays and collections.<br><br>I found on <a href="http://connect.microsoft.com/">Microsoft Connect</a> that someone already added a suggestion to add <span class="blsp-spelling-error" id="SPELLING_ERROR_1">IsNullOrEmpty</span> to arrays on 2005.<br><br>This is not a big discovery, but I have been playing with extension methods in <span class="blsp-spelling-error" id="SPELLING_ERROR_2">Orcas</span> and they are so nice!<br><br>What if I define this?<br></p> <p><span style="color: #000080">public static class IEnumerableExtensions<br>{<br> public static bool IsNullOrEmpty(this System.Collections.IEnumerable source)<br> {<br> if (source == null)<br> return true;<br> else<br> return !source.GetEnumerator().MoveNext();<br> }<br>}</span> <p>Once you import the appropriate namespaces, all these things are possible:</p> <p><span style="color: #000080">string a = null;<br>Console.WriteLine(a.IsNullOrEmpty()); </span> <p><span style="color: #000080">var b = new Dictionary();<br>Console.WriteLine(b.IsNullOrEmpty()); </span> <p><span style="color: #000080">MemberInfo[] d = MethodInfo.GetCurrentMethod().DeclaringType.GetMembers();<br>Console.WriteLine(d.IsNullOrEmpty()); </span> <p><span style="color: #000080">var g = from f in d<br>where f.MemberType == MemberTypes.NestedType<br>select f;<br>Console.WriteLine(g.IsNullOrEmpty());</span> <p>I would like to see something like this included in System.Linq.Enumerable static class. Then it would be available to everyone, by default.</p> <p><strong>Update:</strong> I added a more complete entry as a suggestion on <a href="https://connect.microsoft.com/VisualStudio/feedback/ViewFeedback.aspx?FeedbackID=287482">Microsoft Connect</a>.</p> <p><strong>Update 2:</strong> At the Connect site, <a href="http://blogs.msdn.com/madst/">Mads</a> teaches me why he thinks IsNullOrEmpty as an extension method is really a very bad idea. Basically, using the variable.Method invocation syntax on a method that is meant to work when the variable is null, it is very inconsistent with the instance method invocation semantics one usually gives to this syntax in languages like C# and VB. </p> <p>I still think that the method, probably defined as a static method, would be nice to have on Enumerable (because it is already a well-known place to find methods that apply to the IEnumerable interface). Also, I think there is some void in the definition of extension methods. Its designers think that calling them on null instances should generally throw an exception, so why do I need to check for the parameter and throw the exception myself?</p> Diegohttp://www.blogger.com/profile/06211524340372500555noreply@blogger.com0tag:blogger.com,1999:blog-5930129.post-52039304208127784882007-02-09T17:18:00.000-04:002007-05-12T10:22:52.590-04:00Unindexed Foreign KeysA guy called <span class="blsp-spelling-error" id="SPELLING_ERROR_0" onclick="BLOG_clickHandler(this)"><span class="blsp-spelling-error" id="SPELLING_ERROR_0" onclick="BLOG_clickHandler(this)">Jordi</span></span> <span class="blsp-spelling-error" id="SPELLING_ERROR_1" onclick="BLOG_clickHandler(this)"><span class="blsp-spelling-error" id="SPELLING_ERROR_1" onclick="BLOG_clickHandler(this)">Ramot</span></span>, <a href="http://jlopezra.blogspot.com/2006/02/rdbms-foreign-key-indexing.html">puts it in these words</a>:<br /><br /><blockquote><p>To decide if a foreign key needs to be indexed or not, I follow a simple rule: </p><p>I always/only create an index on a foreign key whether: </p><p>1 - A deletion on the parent table is allowed and it triggers a cascade delete on the child table </p><p>2 - There's need to perform JOIN queries from the parent to the child </p><p>In the first situation, an <span class="blsp-spelling-error" id="SPELLING_ERROR_2" onclick="BLOG_clickHandler(this)"><span class="blsp-spelling-error" id="SPELLING_ERROR_2" onclick="BLOG_clickHandler(this)">unindexed</span></span> foreign key will force a full table scan for each parent record deleted. In the second situation, a lack of the foreign key index in the child table will slow down join queries. </p><p>I rarely find suitable to create indexes on foreign keys in other situations though.</p></blockquote><p>I think the question is not only “to index or not to index” on foreign keys.</p><p>I have been debating this subject with my boss (a hardcore <span class="blsp-spelling-error" id="SPELLING_ERROR_3" onclick="BLOG_clickHandler(this)"><span class="blsp-spelling-error" id="SPELLING_ERROR_3" onclick="BLOG_clickHandler(this)">Informix</span></span> believer) all day. We found that <span class="blsp-spelling-error" id="SPELLING_ERROR_4" onclick="BLOG_clickHandler(this)"><span class="blsp-spelling-error" id="SPELLING_ERROR_4" onclick="BLOG_clickHandler(this)">Informix</span></span> creates indexes on foreign keys automatically, while Oracle, DB2, and <span class="blsp-spelling-error" id="SPELLING_ERROR_5" onclick="BLOG_clickHandler(this)"><span class="blsp-spelling-error" id="SPELLING_ERROR_5" onclick="BLOG_clickHandler(this)">SQL</span></span> Server don't. </p><p>So, why did some engineers decide to go one way and others in the opposite? I think this is an interesting design issue. </p><p><span class="blsp-spelling-error" id="SPELLING_ERROR_6" onclick="BLOG_clickHandler(this)"><span class="blsp-spelling-error" id="SPELLING_ERROR_6" onclick="BLOG_clickHandler(this)">Informix</span></span> takes all responsibility in optimizing <span class="blsp-spelling-error" id="SPELLING_ERROR_7" onclick="BLOG_clickHandler(this)"><span class="blsp-spelling-error" id="SPELLING_ERROR_7" onclick="BLOG_clickHandler(this)">JOINs</span></span> and CASCADING operations on the foreign key. </p><p>Instead, Oracle, DB2 and <span class="blsp-spelling-error" id="SPELLING_ERROR_8" onclick="BLOG_clickHandler(this)"><span class="blsp-spelling-error" id="SPELLING_ERROR_8" onclick="BLOG_clickHandler(this)">SQL</span></span> Server will happily leave the burden of tuning indexes for <span class="blsp-spelling-error" id="SPELLING_ERROR_9" onclick="BLOG_clickHandler(this)"><span class="blsp-spelling-error" id="SPELLING_ERROR_9" onclick="BLOG_clickHandler(this)">JOINs</span></span> and CASCADING operations on your shoulders.</p><p>So, even if an <span class="blsp-spelling-error" id="SPELLING_ERROR_10" onclick="BLOG_clickHandler(this)"><span class="blsp-spelling-error" id="SPELLING_ERROR_10" onclick="BLOG_clickHandler(this)">Informix</span></span> <span class="blsp-spelling-error" id="SPELLING_ERROR_11" onclick="BLOG_clickHandler(this)"><span class="blsp-spelling-error" id="SPELLING_ERROR_11" onclick="BLOG_clickHandler(this)">DBA</span></span> fails to tune the indexes, the database will probably show acceptable performance on JOIN operations.</p><p>Oracle, IBM and Microsoft/<span class="blsp-spelling-error" id="SPELLING_ERROR_13" onclick="BLOG_clickHandler(this)"><span class="blsp-spelling-error" id="SPELLING_ERROR_12" onclick="BLOG_clickHandler(this)">Sybase</span></span> on the other side, apparently decided that tuning was an <span class="blsp-spelling-corrected" id="SPELLING_ERROR_14">nonnegotiable</span> duty of the <span class="blsp-spelling-error" id="SPELLING_ERROR_15" onclick="BLOG_clickHandler(this)"><span class="blsp-spelling-error" id="SPELLING_ERROR_13" onclick="BLOG_clickHandler(this)">DBA</span></span>. However, there are many reasons to want the higher level of control those database engines provide: </p><p>First, each index you create comes with a cost. Not only it will use storage space, but once you created it, the database engine has to maintain it on every table update. </p><p>Also, 98% of all SELECT and UPDATE queries will probably include a WHERE clause or will involve more than one JOIN operation.</p><p>There is also an opportunity for index coverage, meaning that if the index contains certain <span class="blsp-spelling-corrected" id="SPELLING_ERROR_14">columns</span>, some SELECT could be <span class="blsp-spelling-corrected" id="SPELLING_ERROR_15">resolved</span> entirely by reading the indexes, and never touching the real table. </p><p>To get all those benefits at the same time, it is <span class="blsp-spelling-corrected" id="SPELLING_ERROR_16">necessary</span> a composite index that is headed by the foreign key but also includes other columns relevant to frequent queries. </p><p>So, the <span class="blsp-spelling-error" id="SPELLING_ERROR_18" onclick="BLOG_clickHandler(this)"><span class="blsp-spelling-error" id="SPELLING_ERROR_17" onclick="BLOG_clickHandler(this)">Informix</span></span> approach is a winner for the most basic cases, but the higher level of control the other engines give you, could show better performance if tuned <span class="blsp-spelling-corrected" id="SPELLING_ERROR_18">adequately</span> (obviously, my boss won’t swallow that pill!). </p><p>If you want to distill a best practice from this, I think that creating indexes on your foreign keys is a a good first approach, but you should later tune your indexes globally, by using real profiling data. </p><p>Fortunately, for those of us using mostly <span class="blsp-spelling-error" id="SPELLING_ERROR_21" onclick="BLOG_clickHandler(this)"><span class="blsp-spelling-error" id="SPELLING_ERROR_19" onclick="BLOG_clickHandler(this)">SQL</span></span> Server, Index Tuning Wizard exists.</p><p><strong>UPDATE: </strong>You also have to consider how foreign keys are actually implemented. My boss found some articles that mention that in some <span class="blsp-spelling-error" id="SPELLING_ERROR_22" onclick="BLOG_clickHandler(this)"><span class="blsp-spelling-error" id="SPELLING_ERROR_20" onclick="BLOG_clickHandler(this)">RDBMs</span></span> foreign keys are internally implemented as "pointer chains".</p>Diegohttp://www.blogger.com/profile/06211524340372500555noreply@blogger.com0tag:blogger.com,1999:blog-5930129.post-39382302399417155382007-02-03T11:40:00.000-04:002007-02-03T18:55:50.829-04:00Help Find Jim GrayI know I don't manage any significant traffic here, but anyway: If you know of Jim Gray and how he has been missing in the sea from last Sunday, there is a way you can help in finding him.<br /><br />The Coast Guard already called off their search effort and so friends and colleagues <a href="http://www.theledger.com/apps/pbcs.dll/article?AID=/20070203/ZNYT01/702030487/1001/BUSINESS">have taken the challenge</a>. <br /><br />Amazon set up a job in their site <a href="http://www.mturk.com/mturk/startiterator?iteartorSearchSpec=HITGroupSearch%23T%231%2310%23-1%23T%23%21keyword_list%212%21rO0ABXQAA0ppbQ--%21Reward%216%21rO0ABXQABDAuMDA-%21%23%21NumHITs%211%21%23%21">Mechanical Turk</a>. So you can go there, login to you Amazon account and start visually scanning recent satellite images of the search area.<br /><br />Update: You can <a href="http://www.allthingsdistributed.com/2007/02/help_find_jim_gray.html">go read</a> on Werner Vogel's blog how they do it. <br /><br />Please, join!Diegohttp://www.blogger.com/profile/06211524340372500555noreply@blogger.com0tag:blogger.com,1999:blog-5930129.post-14633890048781130442007-01-30T12:24:00.000-04:002007-09-04T10:11:33.065-04:00Jim Gray Is Missing in the Sea<p>Just found via <a href="http://scobleizer.com/2007/01/30/jim-gray-one-of-microsofts-smartest-is-missing/">Scoble</a> that Jim Gray has been missing in the sea since Sunday.</p> <p>I admire Jim Gray, and I hope he will return safe soon. I am also thinking that I really have and want a chance to meet him in person, but this can wait.<br><br>Update 1/31: <a href="http://www.sfgate.com/cgi-bin/article.cgi?f=/c/a/2007/01/31/MNGPMNRVD137.DTL&hw=jim+gray&sn=001&sc=1000">No news</a> is sad news. I do hope he is still alive, but my mind keeps trying to imagine what actually happened.</p>Diegohttp://www.blogger.com/profile/06211524340372500555noreply@blogger.com0tag:blogger.com,1999:blog-5930129.post-24053667772458183852007-01-11T18:58:00.000-04:002007-01-11T19:19:32.585-04:00Vacations, Sweet VacationsI have been on vacations since December 21st, visiting family and friends in my <a href="http://en.wikipedia.org/wiki/Mendoza%2C_Argentina">hometown</a>. I am happy to see that most people here are doing better than two years ago. I haven´t met as many friends as I would have liked, but this is because many of them are out of town on their own vacations.<br /><br />Besides all the happiness that is being here, watching my son interact with his grandfathers and grandmothers, etc., on January 3rd I had some wonderful news:<br /><br />I got an offer from <a href="http://www.microsoft.com/">my favorite company </a>to work with <a href="http://blogs.msdn.com/adonet/">my favorite technology</a>. Still, I can hardly express how it feels!Diegohttp://www.blogger.com/profile/06211524340372500555noreply@blogger.com0tag:blogger.com,1999:blog-5930129.post-90608714236754577952006-12-14T16:36:00.000-04:002006-12-15T00:39:33.191-04:00Broken Windows (Part 1)<p>Today at work, someone I respect much told me that I write very well (in Spanish) and that I am both inspiring and motivating to others. She also said that those were very rare features in a developer. </p><p>Seldom I have been aware of this. Actually I believe that the real underlying fact is that I cannot go to work every morning if I am bored of it and I don't feel at ease. I think I could not keep a boring job for a week, just for the money. So, when I am becoming bored, I have to do whatever it takes to make it fun again. People that see me, understand very quickly that I work for the fun, and that I love what I do. I don't know for sure, but when I see someone that seems to love what she or he does, it helps me keep my own fire alive.</p><p>During Tuesday afternoon, I talked on the phone with a very special group of people. Now I wish i could have a second chance to tell them: </p><p><strong>See guys? Someone I work with thinks I am inspiring and motivating! John? Are you there? At least you have my blog address... :)</strong></p><p>Well, the writing my office mate was referring to is an email I wrote to many developers in our organization about a rather basic topic that we needed to reinforce. I have been involved a lot in code quality initiatives lately. I need to translate the text in order to publish it, so please wait until my next post.</p>Diegohttp://www.blogger.com/profile/06211524340372500555noreply@blogger.com2tag:blogger.com,1999:blog-5930129.post-1165823393827307852006-12-11T03:43:00.000-04:002006-12-12T10:37:58.216-04:00Jon Udell gets assimilated! ;)I have been a fan of Jon for 18 years. This is great news. I think he will have a very important role in the ongoing Microsoft change.<br /><br />With Rory and Jon on board, <a href="http://on10.net/Blogs/jeff/jon-udell-is-joining-our-team-at-microsoft/">Channel 9/10</a> looks like the dream team.<br /><br />Now, let's see how I do it this week...Diegohttp://www.blogger.com/profile/06211524340372500555noreply@blogger.com0tag:blogger.com,1999:blog-5930129.post-1165817473953043532006-12-11T01:22:00.000-04:002007-06-15T01:41:52.358-04:00Transactional File System in Windows Vista (Part 2)In <a href="http://diegov.blogspot.com/2006/12/transactional-file-system-in-windows.html">my last post</a> I tried to describe what I think is the single most important issue with the implementation of TxF (and TxR).<br /><br />Now I want to explain how I think the pre-existing file name-based APIs could have been changed (or could still be changed in future versions) to allow for opting-in Transactions for virtualy any code that works with files, including things like System.Data.DataSet.WriteXml(String).<br /><br />I take this just as an exercise for my mind, as I am almost sure that someone thought about this solution, but then discarded it for a reason I cannot discern.<br /><br />First of all, let’s say that all code running inside or on top of Windows, when it needs to access the file system, ends up invoking on of a relatively small set of Win32 APIs. Of those, some are file-name based (like CreateFile, DeleteFile, SetFileAttributes, etc.) and some are file-handle based (like GetFileSize, ReadFile, WriteFile, etc.).<br /><br />Second, you always have to call first a name-based API to get a handle you can use with a handle-based function.<br /><br />Third, no new Transacted versions of handle-based APIs were created. Instead, handle-based APIs get transactional behavior only if the handle passed is already associated with a transaction.<br /><br />(Still, it bugs me why some name-based APIs were not replicated but instead behave like in the beta 2 model, becoming transactional depending on the ambient transaction).<br /><br />So, you only signal that want to participate in an existing transaction at the time you call name-based APIs. In the current model, the way to do it is calling the transactional version of the function (like CreateFileTransacted, DeleteFileTransacted, SetFileAttributesTransacted, etc).<br /><br />But what all those functions have in common in the first place, is that they receive a file name as a parameter!<br /><br /><em>(<strong>Edit: </strong>What follows is my proposition, not how TxF/TxR works in Vista. Somehow, after some editing I got the text wrong. )</em><br /><br /><em>What if we change rules a little:</em> If the file name is prefixed with a moniker like, for instance “txf:” or “txfile:”, then it becomes transactional. You can understand it as designating a new namespace for TxF, one that points to the same <em>file system</em> store, but behaves differently.<br /><br />There are already <a href="http://msdn.microsoft.com/library/default.asp?url=/library/en-us/fileio/fs/naming_a_file.asp">many rules</a> about file names. However, for me it makes sense to add just one more in this case. Of course the exact form of the prefix is not important (NTFS seems to prefer other kind of prefixes).<br /><br />By this plan, calling CreateFile(“txf:foo.txt”…) would be equivalent to calling CreateFileTransacted(“foo.txt”…). Something similar could be done with TxR.<br /><br />I think this change would integrate perfectly with the model that shipped in Vista, meaning that you could mix and match calls to the Transacted functions with calls to the “normal” functions with the moniker prepended to the file name.<br /><br />Also, existing client code and code “hidden” across the programming stack would not need to opt-out. It would be automatically be not transactional because its hard-coded file names (or registry key names) would not contain the transactional moniker.<br /><br />But the real benefit would be that this could enable us to do things like myDataSet.WriteXml(“txf:foo.xml”) or [insert any other function that takes a file name as a parameter](“txf:bar.ext”), without waiting for a revision of them. Virtually all the programming stack could enjoy the benefits of TxF, without requiring modifications.<br /><br />I can only think of a few functions that try to parse the file name and that could fail on the presence of the prefix, but I think those would be rare exceptions.<br /><br />What do you think? If you like it, you can vote for this as I entered <a href="https://connect.microsoft.com/VisualStudio/feedback/ViewFeedback.aspx?FeedbackID=244584">a closely related suggestion</a> to the .NET Framework team in Microsoft Connect.Diegohttp://www.blogger.com/profile/06211524340372500555noreply@blogger.com0tag:blogger.com,1999:blog-5930129.post-1165386579903728112006-12-06T02:29:00.000-04:002006-12-11T01:22:39.000-04:00Transactional File System in Windows Vista<p>In November, 1998, Microsoft Transaction Server was about a year old and SQL Server 7 was just arriving. I had at hand the task of coding a small CRM-like application in Visual Basic 5. Among other features, it had to upload unstructured documents and keep them linked to rows in a database. </p><p>I had one important decision to make: Should those documents be stored in the database itself or in the server file system? </p><p>SQL Server 6.5 had a lot of limitations with its lack of row locking and some performance issues with BLOB columns. </p><p>On the other side, the file system lacked transactional capabilities, and I lacked the ability to create a Compensating Resource Manager. </p><p>A transactional file system would have been super useful. </p><p>In November 2006, eight years later, Windows Vista is available. Transactions were introduced as a new feature of NTFS, named TxF. The Windows Registry is also getting support for transactions in Vista, under the name of TxR.</p><p>Before TxF, for instance, if you wanted to get ACID-like behavior from multiple file system operations, you could, but you had to fiddle a lot with temporary files, renaming, etc. With TxF you just issue something like a "begin transaction", then do your stuff in NTFS, and last, you commit or roll back the whole thing. </p><p>This way, TxF pushes best practices under the rug, and pushes the developer one level of abstraction up regarding files.</p><p>Surendra Verma, Developer Manager in the CFS group, <a href="http://channel9.msdn.com/ShowPost.aspx?PostID=259679">explained how TxF/TxR works in Channel 9 some months ago</a>. But it is interesting to note that after the video was recorded, there were major design changes to TxF/TxR.</p><p>As Jim Johnson explains in this <a href="http://pluralsight.com/blogs/jimjohn/archive/2005/04/27/7811.aspx">first</a>, <a href="http://pluralsight.com/blogs/jimjohn/archive/2005/09/13/14803.aspx">second</a> and <a href="http://pluralsight.com/blogs/jimjohn/archive/2006/08/31/36819.aspx">third</a> posts, from Beta 2 to RC1, TxF API changed from "implicit transaction enlistment" model that worked with the existing Win32 file APIs to a more explicit model for which new "Transacted" versions of some APIs were added.</p><p>In the first version, you just did something like:</p><p><span style="font-family:Courier New;"><strong>EnterTransactionScope();<br />// do whatever file work with your favorite file APIs<br />ExitTransactionScope();</strong></span></p><p>Everything you did in the middle got automatically enlisted in a thread specific ambient transaction.</p><p>In the new model, you have to do something like this (some function names were invented):</p><p><span style="font-family:Courier New;"><strong>hTransaction = GetTransactionHandle();<br />hFile = CreateFileTransacted(... hTransaction ...);<br />// do whatever, but now using new *Transacted APIs<br />CloseHandle(hTransaction );</strong></span></p><p>The complete listing of APIs that were affected by TxF is <a href="http://msdn.microsoft.com/library/default.asp?url=/library/en-us/fileio/fs/win32_functions_changed_by_transactions.asp">here</a>.: </p><p>If you take a look at it, all the APIs for which a new Transacted version were created are file name-based. Besides, some existing APIs were updated and are now transaction aware, meaning that they acquire transactional behavior in the presence of a file handle that is associated to a transaction (for file handle-based APIs) or in the presence of a thread level ambient transaction (for yet another group of file name-based APIs). </p><p>The reason Microsoft change models, as explained by Surendra in the discussion of the video in Channel 9, is that the more simple original version, had a major drawback: </p><p>Between any pair of EnterTransactionScope()/ExitTransactionScope(), every single file or registry operation made by any code, even code lost in the middle of the programming stack was automatically and forcefully enlisted in the ambient transaction, acquiring a behavior that was often not intended at the time such code was created. </p><p>You could not opt-out.</p><p>So, if implicit transactions means that current code will break or misbehave, it is good that they abandoned this path. </p><p>On the other side, the main tradeoff of the new version, in my opinion, is that it is "too explicit":</p><p>Only those new APIs and those that have been changed will get transactional behavior. So, the hundreds, if not thousands, of higher level APIs that somehow affect the file system, won't get the possibility of having transactional behavior until the whole stack gets updated.</p><p>Now, you cannot opt-in.</p><p>For a .NET developer like me, this means that I have to use a lot of interop, or wait until new versions of System.IO.FileStream, methods like System.File.Delete, and even that the brand new APIs in System.IO.Packaging get revised to include the option of using transactions.</p><p>I have been thinking of a deceptively simple change they could do to the existing file name-based APIs, that could solve this issue. I must be missing something, or they would have implemented it on Vista.</p><p>I tried to discuss my idea con Surendra, but he is probably having vacation after shipping Vista :)</p><p>I will try to explain the idea in my next post...</p>Diegohttp://www.blogger.com/profile/06211524340372500555noreply@blogger.com0tag:blogger.com,1999:blog-5930129.post-1164575265699836422006-11-26T17:07:00.000-04:002006-11-26T18:15:20.306-04:00Arbitrary precision types for .NET (and some musings)<p>I came up today to <a title="Arbitrary length Integer/Arbitrary precision Double Classes [Ari Weinstein]" href="http://blogs.msdn.com/bclteam/archive/2006/07/20/672818.aspx" target="_blank">this article</a> that describes two new arbitrary precision types for .NET. One is an Integer, and the other a Decimal (I think it is not correct to call it a Double as in the title).</p> <p>Although I don't often use arbitrary precision math, this is a welcome addition. It is also interesting to think how nicely such a feature can integrate in the existing framework, given generics and the unified type system.</p> <p>Speaking of which... I found recently that the Parse and TryParse methods that numeric value types share do not belong to a common generic interface, reducing its usefulness in some scenarios. </p> <p>I hope we will someday see something like IParseable<T> together with IComparable, IEquatable, IConvertible and IFormattable. </p> <p>And what about the MinValue and MaxValue constants? Shouldn't they belong to a common type?</p> <p>There is also this internal class called Number that seems to hold the implementation of methods of every numeric type. I wonder if they could somehow introduce a Number class as a common ancestor without breaking value types rules.</p>Diegohttp://www.blogger.com/profile/06211524340372500555noreply@blogger.com2tag:blogger.com,1999:blog-5930129.post-1161058911967062292006-10-17T00:08:00.000-04:002006-10-17T00:34:06.470-04:00AshamedI got the call, I went there, but then I disappointed myself. <br /><br />No matter if I was trying to create a recursive function that wouldn't suck as much as the usual f(n)=if(n<=2,1,f(n-1)+f(n-2)). I should have just written this in a couple of seconds as I did now! Or maybe the more correct f(n)=if(n<2,n,f(n-1)+f(n-2)) (that is still only correct for natural numbers, but doesn't check it of course). <br /><br />But I was too much out of shape. <br /><br />How can this have happened to me? It should have had some impact that as a kid I was fascinated by the Fibonacci Series, Tartaglia's (Pascal's) Triangle and the Golden Number. I even discovered (or reinvented) myself the Binet formula when I was a teen. What for then? Maybe I should have applied for my dream job 15 years earlier.<br /><br />I have never felt so uneducated before. I should probably go back to the University and get a Master in Computer Science, if my brain can still take the challenge. I cannot forgive myself for getting stuck with something so simple as a recursive Fibonacci.Diegohttp://www.blogger.com/profile/06211524340372500555noreply@blogger.com0tag:blogger.com,1999:blog-5930129.post-1155522006970949492006-08-13T22:20:00.000-04:002006-08-13T22:20:33.316-04:00Windows Live Writer<p>Hmmm... So far it feels nice :) <a href="http://windowslivewriter.spaces.live.com/">Good job!</a></p>Diegohttp://www.blogger.com/profile/06211524340372500555noreply@blogger.com0tag:blogger.com,1999:blog-5930129.post-1154495777001686282006-08-02T01:14:00.000-04:002006-08-02T01:16:17.026-04:00Rory's Best Post EverDude! Congratulations! I hope you are living the best time of your life (so far) too.Diegohttp://www.blogger.com/profile/06211524340372500555noreply@blogger.com0tag:blogger.com,1999:blog-5930129.post-1154489833212059762006-08-01T23:24:00.000-04:002006-08-02T01:32:42.956-04:00Spaces Comming to Live Right NowI remember I was one of the first to get my own MSN Space that night, and as a result, I got trapped in a bug and could not use the the site for some time (I don't use it a lot anyway) .<br /><br />Now I think tonight I catched them in the act again. They are transitioning MSN spaces to their new Live home, and they still have some work to do.<br /><br />Take a look at how a famous space (<a href="http://rayozzie.spaces.live.com/">Ray Ozzie's</a>) looks like right now:<br /><br /><a href="http://photos1.blogger.com/blogger/1146/254/1600/Ray%20Ozzie%20Live%20Space.png"><img style="DISPLAY: block; MARGIN: 0px auto 10px; CURSOR: hand; TEXT-ALIGN: center" alt="" src="http://photos1.blogger.com/blogger/1146/254/400/Ray%20Ozzie%20Live%20Space.png" border="0" /></a><br /><br />Update: Around 40 minutes later, everything is comming together nicely. No more missing images or CSS. It is amazing how the new look captures the escense of the MSN space theme you choosed, but still looks ver LIVE like.Diegohttp://www.blogger.com/profile/06211524340372500555noreply@blogger.com0tag:blogger.com,1999:blog-5930129.post-1153797284577071722006-07-24T23:12:00.000-04:002006-08-02T01:31:00.926-04:00Rory Blyth: Blogging is StupidJust read this gem:<br /><br /><blockquote>One of the examples of a risk that Alfred brings up in his post is my tendency to write about my depressive nature and the drugs I take to try and undo the crap job nature did on wiring my neurons. I’ve been told similar things by some of my friends in the “blogosphere.” They tell me I’m nuts for posting about my mental problems. I agree. But that’s part of being nuts, so what’s the confusion about?</blockquote><br />Is it stupid or not? Well, I guess it is stupid, but it is more stupid when you cannot write like him :)Diegohttp://www.blogger.com/profile/06211524340372500555noreply@blogger.com0tag:blogger.com,1999:blog-5930129.post-1153797080884889382006-07-24T22:35:00.000-04:002006-07-24T23:11:21.333-04:00Fyodor DostoevskyI am very grateful to <a href="http://figuraciones.blogspot.com/">Juan</a> and to Stef for introducing me to the writings of this "crazy" russian genious.<br /><br />The more I read, the less I blog.Diegohttp://www.blogger.com/profile/06211524340372500555noreply@blogger.com0tag:blogger.com,1999:blog-5930129.post-1153794379512464392006-07-24T22:13:00.000-04:002006-07-24T22:35:11.716-04:00ADO.NET Entity Framework on Channel 9Channel 9 published a <a href="http://channel9.msdn.com/Showpost.aspx?postid=217633">very interesting video</a> about Microsoft's O/R Mapping tool that will be included in a future version of ADO.NET. It is actually a remake of a video that was published a couple of months, also in Channel 9, only to be shutted down soon after.<br /><br />As almost every .NET developer does, I use ADO.NET every day, and have been a fan from <a href="http://discuss.develop.com/dotnet.html">the first hour</a>.<br /><br />But as an architect -in a limited sense I must add- I have to deal a lot with both theoretical and empirical aspects of multi-tier development, so I have been looking for the silver bullet for the Data Access Layer for years.<br /><br />Pablo Castro, Technnical Lead of ADO.NET (and a native from Buenos Aires!) was very nice answering the comments <a href="http://channel9.msdn.com/ShowPost.aspx?PostID=219017#219017">I posted to Channel 9</a>. I will reproduce my own writting here, but you can read his very interesting answer <a href="http://channel9.msdn.com/ShowPost.aspx?PostID=219272#219272">directly on the site</a>.<br /><blockquote><p>As I mentioned before, one of my great concerns is how team development will look like with the Entity Framework. I took some time to detail my thoughts: </p><p>First, many real life projects are partitioned in modules, so their data layers are partitioned likewise. </p><p>Often, there are sets of tables that are used exclusively in each module, and a set of tables that are common to all. Yet, there are some tables that are reused in more than one application (typical examples are security, navigation, etc). </p><p>Besides, building a useful data layer is not done in one step nor does it take a single day. It is more often an evolutionary and error-prone process in which a programmer “imports” objects from the database each time he/she realizes they are mentioned in the specification. </p><p>During this process, errors that affect maintainability (duplications, improper use of naming standards, etc.) are very usual. </p><p>So, here is a short list of features that I would like to see in the Entity Framework (some are actually hard requirements). Of course I ignore if any of these are already included: </p><ol><li>Partitioning of the conceptual model in multiple files and assemblies. </li><li>Referencing and extending (entity inheritance) between entities defined in separate files and assemblies. </li><li>Creating reusable “libraries” containing entities and mappings that can be reused by different modules or different applications. </li><li>"Incremental" reverse engineering of databases (I think this one is already in the graphical design tool). </li><li>Support for basic refactorings (unification, replacement, renaming, etc). </li><li>Very readable and maintainable XML (it should be easy to merge two files with a source code comparison tool). </li><li>Efficient and easy serialization of entities and entity sets outside the database. </li><li>Separation of the conceptual model from the persistence logic (take a look at what Steve Lasker does with typed datasets). </li><li>A migration tool for typed datasets XSDs. </li><li>A degree of resiliency to some schema changes. </li></ol><p>If you can answer any of my doubts, I will be grateful. My boss is pressing me to evaluate O/RM products, and I am telling him to way for your framework everyday :).</p></blockquote><br /><br />All in all, I am very enthusiastic about this new piece of technology. I think it could make my life and the life of many other developers a lot easier and also more interesting!Diegohttp://www.blogger.com/profile/06211524340372500555noreply@blogger.com0tag:blogger.com,1999:blog-5930129.post-1153279982392509312006-07-18T23:31:00.000-04:002006-07-18T23:33:02.416-04:00Microsoft buys Winternals for the talentI am glad for Mark Russinovich and for Bryce Cogswell, but I am even more happy for the future of Windows. These guys really create amazing software.<br /><br />First improvment I hope to see in Windows is a "Suspend" option in the process view of Task Manager... It is an obvious one, isn't it?Diegohttp://www.blogger.com/profile/06211524340372500555noreply@blogger.com0