Posts in category microsoft - page 6

What being open means to Apple & Microsoft


Former Apple engineer Jens Alfke believes Apple’s external image has been polished until featureless. The restrictive staff blogging policies, the veil of secrecy around future plans and a carefully orchestrated three-person spokes-team of Jobs, Schiller and Ive lead to a very impersonal closed business.

It certainly wasn’t always this way. The original Mac team appeared in Rolling Stone magazine with credit in about boxes, a practice that was continued at NeXT but abolished by Mac OS X Beta. Jobs makes regular comparisons between engineers and artists and touted individual thinking in the Think Different campaign and artists like recognition with signatures on art and credits on film.

Conversely Apple’s Mac OS X operating system is built on open software and standards. The kernel is derived from open elements bundled up as Darwin which Apple provides back along with compilers, debug tools, programming language, command line tools, Bonjour, device driver kit and a bunch of drivers. All are open.

The web rendering technology in Safari (WebKit based on KHTML) is also open and changes rolled back to the communities often reveal unannounced insights into Apple’s plans (e.g. Safari for Windows).

And yet how many engineers write or talk about Apple? Do you know the names of any product managers? Could you find any out with Google? (LinkedIn doesn’t count ;-)

These aren’t academic questions, what if you have a great idea for a feature you’d like to see added? How can you discuss how a product could evolve to fit your needs? What about a simple bug report or advanced access to technology? (The answers are “send it to and don’t hold your breath”, “you can’t” and “join the developer program”)

Heaven forbid you do actually find out what their plans might be – you could find yourself talking to their lawyers like the ill-fated ThinkSecret site that featured rumors, speculation and the occasional insider info.


Jens makes a passing mention to Microsoft’s relaxed blogging policies.

Microsoft is a company that rarely provides the source, never ships or builds upon existing free software and yet not only discusses plans and road-maps but actively solicits feedback in the design process through conferences, user groups, forums, mailing lists and even on-site review teams. Employees such as Scott Guthrie and Brad Abrams have become quite well known within .NET communities often being the first to break announcements and provide quick feedback through their blogs.

The center of this effort is engineering thanks to sites like Channel 9 providing regular interviews, Microsoft Research providing experiments to play with and CodePlex hosting open projects.

But they aren’t the only ones reaching out.

Microsoft’s HR & recruiting team and individuals are also putting up interesting insights and thoughts on how the company operates and head of the Xbox Live! is so active in this area that the name Major Nelson is known to any serious 360 owner.

Being open

How strange that Apple embraces open technologies yet keeps communication closed and Microsoft’s technologies are still quite closed yet communication is very much open.

What does it mean to be open and where will each company’s approach lead them?


When SQL Server replication eats disk space

Part of my job involves revising our SQL Server architecture. My plan includes the addition of a read-only reporting SQL pair for non-critical inquiries and reports. This allows the heavy and unpredictable load from reporting away from from the primary SQL pair responsible for critical operations (shipping orders).

We utilized SQL Server’s publisher-subscriber replication on the required databases which, given their legacy nature, had some cross-database dependencies that were added without due consideration.

The reporting SQL pair needed to be rebuilt so the subscriptions were removed whilst they were being rebuilt. Within three days our primary servers were out of disk space and we had a problem that needed to be solved quickly.

Our first step was to remove the new publications. This released a small amount of space in the distribution database (now 100GB) and allowed the replicated databases logs to be truncated and shrunk. We now had a little breathing room.

One publication would not remove in a reasonable time (10+ hours) and even trying to remove individual articles caused a slow-running operation sat on PAGEIOLATCH_SH (which Microsoft describe as waiting on disk IO). A quick investigation revealed a complex query on internal system tables with no indexes – presumably for fast insert performance.

In the mean-time this slow-running operation would block the replication agents which are also used by IBM’s DataMirror product to replicate data back to the iSeries (AS/400) in real time. This blocking would prevent us shipping orders and was therefore unacceptable.

The database in question is a home-grown data warehousing application rebuilt each night and on reflection was a terrible idea to replicate using this method. Each night it generated over twice it’s own size in replication activity. Given we have a 72 hour replication retention and the size of the database that alone was enough to wipe out the disk space on our server within 3 days.

Reinitialize All Subscriptions was the magic bullet which ran in just a minute allowing the publication to be subsequently instantly deleted. The replicated databases transaction logs were then truncated and shrunk giving us ample breathing space on the storage front.

The distribution database was still huge but heading into Replication Monitor’s Common Jobs and starting the Distribution clean up: distribution job was our final step.

The job might take all day but did not block any other replication activity :)


SQL Server replication blocking on cleanup job

For some time my primary workplace has been having a problem with SQL Server replication delaying for several minutes at a time which is surprising given the 12GB of RAM and quad processor hardware behind it.

Activity Monitor showed a number of processes on the distribution database used for SQL Server’s replication were blocked by another process which was in turn blocked by a SQL Agent – TSQL Job executing on the distribution database.

This process’s job GUID that did not match any of our jobs and appeared to be an internal job that regularly executes:

EXEC dbo.sp_MSdistribution_cleanup @min_distretention = 0, @max_disretention = 32760

This would sit on the PAGEIOLATCH_SH status for over 10 minutes at a time most most likely because it was operating on an MSrepl_commands table with 3.4 million records and a MSrepl_transactions table of 600k records.

The default max_disretention is 72 (hours) and specifies how long to hold the replication history for. This means if a subscriber is down for longer than this period it will loose transactions.

Quite how this system obtained the value 32760 (3.7 years) is a bit of a mystery and I’m assuming that it actually uses a lower value as there have been more than 3.4 million updates in that period.

Setting this value to 84 hours (3.5 days) in Management Studio’s Replication > Distributor Properties > Transaction Retention soon dropped those large MS tables down to 406 records and 300 respectively and resulted in sp\_MSdistribution\_cleanup executing in a couple of seconds.

It’s a very specific scenario but perhaps this blog post will be of use to somebody else!


Returned from Redmond

It’s been an overwhelming few days listening and interacting with bright people from the .NET community and within Microsoft itself (wish I could say more but I can’t). Here are just a few of those names, I wish I knew them all but I was so busy listening to what they had to say I often forgot to ask for a card:

And from the Microsoft side of the fence

Congratulations go out to Rob Conery who is now getting paid by Microsoft to work on SubSonic!

How cool is that!