The book is divided into 34 small chapters, each based on insightful stories based on in-the-trenches experiences. Lots of people-books offer fluff: "be nice to all your coworkers", "work hard", "always brush your teeth so your bad breath doesn't alienate your coworkers", etc... Michael bypasses the obvious and gets to the good stuff. Some of the big points I took away:

Blunt Truths

1. "Your manager is not a manager until they've participated in a layoff." (pg. 15)
2. "If you're sitting in a meeting where you're unable to identify any players, get the hell out." (pg. 23)
3. "Remember that for every person on the team who has a strong opinion regarding the decision, there are probably four other coworkers who just want someone to make a decision so that they can get back to work." (pg. 28)
4. "you aren't a company until 1.0 is done." (pg. 77)
5. About reacting vs. thinking, and being too busy: "when you're busy, you're not thinking, you're reacting." (pg. 83)
6. About "Malcolm Events" - "Seemingly insignificant events that are intent on screwing you in an unlikely way." (pg. 93) "The only way you're going to learn to identify potential Malcolm events is by going through some horrible, horrible experiences." (pg. 96) Part of avoiding these events is clear and tough communication that most people want to shy away from, such as team status reports that say "We're not doing Phil's favorite feature."
7. "nothing gets everyone's attention like a deadline." (pg. 107)
8. About finding the anchor in a meeting - "Just wait for someone to say something controversial and see who everyone looks at." (pg. 148)
9. "Like it or not, your boss has as much effect on your career as you do" (pg. 163)
10. "A reorg isn't over until someone important has printed out a new organizational chart and presented it in front of the entire company." (pg. 174)
11. About outsourcing your job - "You could be outsourced because your job is so richly defined that it can be documented and explained to any reasonable professional on the planet..." (pg. 179) "Jobs that can be 'well specified' are being shipped offshore." (pg. 183)
12. "A micromanager does not trust." (pg. 189)
13. "Guy who knows the people are the business." (pg. 190)

Other misc quotes

1. "you are not talking to a person when you talk with your manager; you are talking to the organization." (pg. 11)
2. "understanding your manager's place in the political food chain is the trickiest because you're often not in the meetings where he is interacting with his superiors." (pg. 14)
3. "In any freakout, there is normally a very noisy preamble which is designed to get your attention." (pg. 18)
4. "your job is not just management of people, it's management of information." (pg. 105)

That sort of stuff bothers me too, but I'm still a big fan of good process. What I realize is that I essentially view "process" as "developer infrastructure enhancements". I think of process as helpful things like automation, unit tests, code generation, proper tools, CI builds, checkout and install scripts, etc...  Devs just want to get the job done, and good process assists them in doing that, it doesn't burden them with ivory-tower taxes. If your process code-generates the data access layer, then the dev need not do all that manual ADO.Net plumbing code, and hence the dev gets the job done faster.

Sure, it's semantics - "process" vs. "infrastructure enhancements", but semantics are important because it affects how a thought is communicated to other people, and people are important.

• Close VS, delete the *.suo file, and try to re-open. I'm sure there's a reason why,

2 seconds later, VS was up and running.

Here's a brain-dump:

1. Condition your mind
   1. Until you are employed, your job is to find a job. Prepare to spend at least 4 (maybe 8!) hours a day actively pursuing job opportunities.
   2. Furthermore, you're not seeking to "get a job", you're looking to "add value" to a company by solving problems in a technical field that you're passionate about.
2. Prepare
   1. Set up a linked-in account. This is an effective and professional way to keep track of people you meet.
   2. Make the equivalent of a business card that you can hand out as you meet people. Even a card saying something like "Joey Finklestein, my-email, 'Technology Specialist'" is good. The goal is to have your contact info easily available.
   3. Get your resume ready. make sure it downgrades to plain text in case you need to dump it into some online text area. I personally don't think resumes are the biggest deal. Yes, everything counts. But if you're blindly submitting your resume online, you're one among thousands, and it probably won't matter (sorry). If you meet someone in person, the impression you make will probably dwarf any wordsmith-ing on your resume. If you've actually got even a phone screen, the resume has already been sufficient.
3. Network. Meet people.
   1. Especially if you live in a larger city (like Chicago), prepare to go to a user group meeting at least once a week. For example, Chicago has dozens of user groups (LCNUG, ALT.Net, CNUG, SQL groups, IT, SharePoint, TFS, Design, etc...) Just google it, there's probably a group. Even if the group isn't exactly on target, go to the closet-related thing. Try to meet at least 3 people. Talk to them, ask them what they do, get their business card, give them your business card. Often user groups have recruiters who are trying to fill positions - talk to these people. Even if their position isn't an exact match, they may know of another position, they may have a position that frees up later, or they may just offer you good advice. Plant seeds.
   2. Keep in touch with your graduating class. Maybe they have leads.
   3. Go to job fairs - most community colleges offer these on a regular basis.
4. Start a corporate and professional online presence.
   1. Contribute to online discussion boards (like http://stackoverflow.com/)
   2. Contribute to an open-source project (check out CodePlex.com)
   3. Consider writing some articles (either start your own blog, or contribute on a free site like CodeProject. Even if you're just writing simple articles like "Joey's C# 101 tutorials", it's still beneficial. It tells employers that (1) you're motivated, (2) you can write (non-tech skills are a great asset), (3) you're pro-active enough to write. It will also make you more confident after you've explained things in an article. Try to write at least two short blogs, or one longer article, every week. Even if you're "not the writing type", employers want people who can write, and having a repository of your articles shows them, as opposed to summary statements at the top of a resume that say "has good communication skills". You can then also list your blog on your resume.
5. Continual Education - industry is a different beast than academia. The CS degree is great, but that's the beginning, not the end.
   1. Prepare to spend at least an 1 hour a day reading blogs that are relevant to the job you seek. Find who the top bloggers are in your field of interest, and read them. Probably get an RssReader.
   2. Read the job postings on online sites like Monster, Dice, HotJobs, etc.... You want to make sure you know all the buzzwords, and see what employers are asking for.
   3. Do charity projects. Many charity groups could really use the free help. Offer to do a tech-related project (assist with their website, do a data migration, write a tool to help them do some task). It doesn't pay cash, but it does pay in experience and relationships.
6. About applying...
   1. Ideally you want to meet someone in person (like at a user group). Next best thing is to meet someone in the company who can provide a referral.
   2. If you do apply online (with no personal reference), don't put all your eggs in one basket - apply to several companies. But don't spam Monster. Perhaps submit to a few companies each day, but not more than 10 companies at once. If you don't hear back from a company within 5 days, move on. Many companies send out an automated "we received your resume note", and then only personally follow-up if they're interested.

Case 1: Regular expressions

Take an email validator, something like so:

public static bool IsEmail(string s)
{
    return System.Text.RegularExpressions.Regex.IsMatch(s,
        @"\b[A-Z0-9._%+-]+@[A-Z0-9.-]+\.[A-Z]{2,4}\b");
}

A single test would give 100% coverage, but obviously there's a lot of other paths to check. Ironically, because regular expressions are often used to validate input data, and it's an in-memory operation (no databases or external files to hit), it's a prime candidate for lots of unit tests to catch all the boundary conditions - as opposed to just the 1 test needed to reach 100% coverage.

Case 2: Single-line expressions

Similar to the previous case, merely calling this method with one set of inputs (say the "less-than" path, such as i1=5 and i2=10), will get 100% coverage. But that wouldn't test the "greater-than" and "equal to" conditions.

public static bool IsGreater(int i1, int i2)
{
    return (i1 > i2);
}

Case 3: Missing Asserts (bad logic)

Even with 100% coverage, it doesn't guarantee that the method logic is correct.

For example, say you've got a CSV-parsing method:

public static string[] ParseCsvString(string strLine)
{
    string[] astr = strLine.Split(',');
    return astr;
}
 

That is tested by:

[TestMethod]
public void ParseCsvString()
{
    string[] astr = Foo.ParseCsvString("a, b, c");
}

This will give 100% coverage. However, there are no asserts, so it's really just showing that the method didn't throw an exception. Even if a developer adds an assert, they need to make sure it's asserting the right thing. Say, adding an assert that the returned array is not null, or has a length of 3, misses the logic that trims the white space after each comma. For example, we want elements like "b" (no whitespace), not " b". In other words, we'd want the ParseCsvString method to loop through each item and Trim() it.

Case 4: Mocking giving a false sense of security

Mocking Frameworks, like TypeMock, are very powerful tools for increasing unit test coverage. These tools allow you to "mock out" a method call within code, such as that database or logging call that would be hard to run in a test method.

While this is great for testing legacy code, it can easily be abused. If every line is mocked out, there's nothing real that's left to test. So while it does get high coverage, if used incorrectly, it becomes meaningless.

I saw a few big take-aways:

• "The journey with others is slower than the journey alone" (pg. 198)
• Quoting Andrew Carnegie, "No man becomes rich unless he enriches others." (pg. 230)
• "If individuals don't possess people skills, they very quickly hit a ceiling in their effectiveness." (pg. 242)
• "Most of us admire and respect people who sustain solid, long-term relationships." (pg. 259)

The book is also filled with good quotes, like:

• Quoted T.S. Eliot as saying "Half the harm that is done in this world is due to people who want to feel important." (pg. 11)
• Quoted someone "the difference between who you are today and who you will be in fie years will be the people you spend time with and the books that you read." (pg. 13)
• "Actions are remembered long after words are forgotten" (pg. 42)
• "People who add value to others almost always do so intentionally."
• "We often expect maturity to come with age, but the truth is, sometimes age comes alone." (pg. 63) - There are young devs who are great, and "experienced" devs who are not.
• "Ultimately the things that bring fulfillment involve others." -  I would find it more fulfilling to use an old technology with friends, where we actually ship the product, then a new "cool" technology by myself.
• "The best way to keep from stepping on other people's toes is to put yourself in their shows." (pg. 73) [I see this continually in the classic software rivalries: managers who want to deliver vs. techies who want to do "cool" stuff; application developers vs. QA, application developers vs. IT infrastructure, etc...]
• "I made a mistake of trying to impress everybody" (pg. 95) - Software engineering is too big, so you can't possibly know it all, so inevitably you'll meet people who know more than you. One of the dumbest thoughts that ever crossed my mind as a younger consultant was "everybody asks me for help, but I don't need to ask anyone else - I must be doing great". Ah, the cluelessness of being young.
• "If you don't like people or don't believe in them, you won't be able to fake it" (pg. 104). Tim's translation: "People can tell when you think they're a moron."
• "Caring for people should precede confronting people" (pg. 107)
• "Quitting is a permanent solution to a temporary problem." (pg 111)
• "Most of the time when you confront people, they will have an emotional reaction." (pg. 114)
• "Most people hate confrontation, but they love resolution." (pg. 115)
• "If you are not honest with yourself, you will not be capable of honest with others." (pg. 126) - Example: If you deceive yourself into thinking the big task will be done in only 4 days, you're going to struggle giving accurate status reports to management.
• "It is more rewarding to resolve a situation than to dissolve a relationship." (pg. 132)
• Quoting someone else: "If you make every game a life-and-death proposition... you'll be dead a lot." (pg. 138)
• Quoting someone else: "A successful man in one who can lay a firm foundation with the bricks others have thrown at him" (pg. 222)
• "People are an appreciating asset only if we are willing to invest in them." (pg. 233)

See also:

BOOK: Making Things Happen, BOOK: Bringing Out the Best in People

• Handles simple postbacks very well
• Independent of server technology (you don't care if it's ASPX, JSP, or PHP providing the responses back)
• Lots of Visual Studio support

• Dies on Ajax because it can't parse JavaScript
• Requires a ton of parsing logic. Granted, there are free Html parsers to help with this.
• Requires the expensive version of Visual Studio for Testers, so not everyone can run on their machines unless management pays $$$ for the licenses.

• Minimizes the parsing
• Free

• Still trouble with JavaScript (if memory serves me right).

• Tests actual UI layout (is the textbox 10 pixels below the button).

• Very brittle with browsers, especially because browsers can resize

• Because it runs the browser itself, it can handle whatever the browser handles, like JavaScript. This is huge in today's Ajax world.
• Free! This is great when you want the entire team to run it on their local machines.
• Active community supporting it

• (I believe, but may be wrong) that because it runs the browser, it's limited to just IE.

Personally, I've seen the best luck with WatiN. Especially in the Ajax age, automated tests need to run JavaScript. I also find that to get a team to adopt a new tool, it's invaluable to let them run it themselves (i.e. anyone can download the open-source tool and run it with management paying $$$ for a license), and to provide tutorials (i.e. WatiN has an active community).

• The code logs incorrect data in an error log file?
• The code displays incorrect values in a label (like a mis-formatted date)?
• If a certain rare case occurs (like a button is pressed at exactly 12:00AM, or an uploaded file size is exactly 1.00 MB), then the code crashes?
• The code has mis-leading names for variables and methods. For example, it has a method "IsNumber" that checks only for integers, or "IsLetter" that allows for special characters? Say the current program calls the method with correct data so that the app never crashes?

In all these cases, say the application essentially "works" and handles the main use cases.

The problem is maintenance. Maintaining and extending code is an expensive part of the total cost of ownership. You could spend hours tracking down a single erroneous line of code. Code that is low quality (tons of copy & paste, misnamed methods, misleading logic, etc...) is going to be a fortune to maintain. On the other hand, certain functional errors that have zero impact on the business can perhaps be documented as "known-issues" (i.e. the month is formatted in a label with a preceding zero like "03" instead of just "3".

I'd say it comes down to the business, and the code is broken if it costs the business more than it should - whether it be via maintenance costs or functional errors that hinder the end users. Perhaps the question isn't as much "is this code broken", but "how can we maximize the business-value of this code?"

1. The devs who would most benefit from unit tests are the devs who are least likely to write them - and vice versa. The star devs, who would write the code correctly to begin with, are also the ones most open to unit testing. Likewise, the low-quality-code-developers who shun testing are the one's whose code could benefit the most from it.
2. Writing unit tests actually saves time - not just in integration testing but also in development - because it stubs out the context, allowing you to immediately jump to the area that needs testing instead of spending 5 minutes setting up the scenario.
3. Developers often punt on unit testing because "my manager doesn't support it", but unit testing is really an encapsulated development detail that doesn't need managerial support (although of course their support is appreciated).
4. Many devs generate the unit tests after they write the code ("those ivory-tower architects said we needed tests"), but tests are most beneficial before you write the code because they force you to think what the code does, and they make it faster to write the code.
5. The same teams who don't want to write unit tests are relieved to have such tests on the code they need to maintain.

This hit home with me as I reach the 5-year mark for my blog. After 5 years of consistent blogging, I've written 430 posts and received hundreds of comments (many of them educating to me). There are a lot of obvious benefits to blogging, but after writing for years, what really sticks out to me is the legacy of blogging. I started blogging at CSC, blogged all through Paylocity, and continue blogging now at CareerEd. Ironically, my blog topics have evolved from hard-core development, to tech lead, to architecture, to more managerial-related tasks.

I can't show people any of the systems I've done (except for the occasional screenshot from a dusty tech doc), but as appropriate, I can share with them the archive of hundreds of blog posts. It helps motivate me to want to write for another 5 years.

Connect the dots - In software programming, we must explicitly spell every step. Sure you can refactor and abstract things out, or use third-party software to spare you from writing it - but somehow, every step must be flushed out in detail. This is great for a robust program, but it will drive real people insane (i.e. managers, business analysts, customers, and executives who write your paycheck). They'll just think you're being clueless, are inexperienced, or a smart aleck. When dealing with actual people, we need to be able to use our background knowledge of the situation to connect the dots.

Don't wait to be asked - It takes effort to ask for something, and people don't like exerting effort, so anytime you can "just know" and do the right thing (perhaps because you know the bigger picture of what they're trying to do), people are going to appreciate it.

• For example, say you see a simple bug in the code. It's a safe change and the release isn't for a while. The savvy developer doesn't need to ask their manager "can I fix this bug" - they just do it (and maybe submit a ticket if their company's process requires that). Often just making the fix can be quicker than asking. (Of course common sense applies, don't go "fixing" mission-critical production code that's out of your scope).
• It seems like only junior resources ask "What can I do to help?"- the senior ones already know.

Of course, it's reasonable to pro-actively notify a manager "I see A, B, and C. I know you're busy, so I'll assume I should start working on 'B' first because of reasons XYZ. Just let me know if you'd like to switch tasks". This lets your manager reply with a 1 word email like "great", and managers like being able to delegate entire tasks with 1-word emails.

"Connecting the dots" and "Don't wait to be asked" are related. Think of this as not needing to be micromanaged.

Translate what people say. There is a world of difference between someone's words and what they actually mean. Whether they've made a simple typo, using poor word choice, or they're struggling to articulate something - it's a big personal win if you can "see the forest through the trees" and know what they mean. You can do this by leveraging context, know where they're trying to go, and having familiarity with what it takes to get there.

• Example: Missing words - A non-technical manager may say "we need to store this dropdown control in the database". They probably mean "..store the value of this dropdown control..."
• Example: Wrong words - A non-technical manager says "We need a bigger machine". They probably mean "we need a better-performing machine"

Summary

You might say "but this isn't fair - I was doing my job to the 'T', I was technically correct" Ah, but life is not fair. You can be right, or you can be happy. And (as I find out the hard way) if you want to be happy with other people, you're eventually need to master many things, including these three.

The book is a casual 400-page read, and certainly lives up the the "for dummies" genre. It offers an overview of starting a small business, from the basics of management to HR to accounting to marketing and economics. I liked the practical tone.

While a book like this doesn't fundamentally change one's view of business, it is useful to get one to casually think about business-concepts during the normal work day. For each project, it prompts me to ask questions like:

• "where does the revenue come from?"
• "who is paying for this project?"
• "how will this project help the business?"
• "can this thing I built actually be marketed?"
• "who are the customers for this product?"

Continually keeping these types of questions in mind also helps a developer relate to business-sponsors, who are the people that ultimately write the developer's paycheck.

• Identifying a business problem such that business sponsors are willing to pay for the product
• Recruit the team to build the project
• Provide the team the tools to develop the app (hardware & software)
• Collecting business requirements
• Coordinating with business partners, such as those providing data that the product will use
• Designing a functional spec
• Creating the architectural and technical designs
• Decide on build vs. acquire (buy, open-source)
• Outsource part of the project
• Managing the project
• Procuring the physical infrastructure that the app is deployed on
• QA testing the app (functional, integration, user-acceptance, performance, etc...)
• Deploying the app
• Write training manuals for the app
• Training support staff and users
• Marketing the app such that people actually use it
• Supporting the app

From start to finish, actually coding for a project may only be 5% - 10% of the total effort. That means that there's a huge portion of the project that is non-coding, and that huge portion can often overcome difficult coding tasks.

For example, say there is a component that is just difficult to program (it's complex, it's big, it occurs outside your expertise, etc...) You could possibly get around coding it yourself by maybe:

• Buying or open-sourcing it (example: use an open-source tool or class library from CodePlex instead of writing it yourself)
• Training the internal end users around using that feature ("we know the website has a bug, but just don't click the browser back button")
• Using project management to get it punted, or out of scope
• Convincing the business sponsor that the feature is not needed ("we don't need to invest all that time making a dancing paper clip assistant")
• Better hardware (example, upgrading hardware for better performance)

The stars who keep delivering successful projects are familiar with this, and they are constantly mitigating challenges in one task by giving up something that doesn't matter from another task.

Sometimes you can solve hard coding problems by just sheer skill and coding right through it. But it's good to be aware of other techniques to work around the problem altogether.

Here's a general strategy to estimate table size:

1. Determine the general schema for the table

Note the column datatypes that could be huge (like varchar(2000) for notes, or xml, or blob)

2. Find out how many rows you expect the table to contain

Is the table extending an existing table, and therefore proportional to it? For example, do you have an existing "Employee" table with 100,000 records, and you're creating a new "Employee_Reviews" table where each employee has a 2-3 reviews (and hence you're expecting 200,000 - 300,000 records)? If the table is completely new, then perhaps you can guess the rowcount based on expectations from the business sponsors.

If the table has only a few rows (perhaps less than 10,000 - but this depends), the size is probably negligible, and you don't need to worry about it.

3. Write a SQL script that creates and populates the table.

You can easily write a SQL script to create a new table (and add its appropriate indexes), and then use a WHILE loop to insert 100,000 rows. This can be done on a local instance of SQL Server. Note that you're not inserting the total number of rows you estimated -  i.e. if you estimated that table will contain 10M rows, you don't need to insert 10M rows - rather you'll want a "unit size", which you can then multiple by however many rows you expect. (Indeed, you don't want to wait for 10M rows to be inserted, and your test machine may not even have enough space for that much test data).

For variable data (like strings), use average sized data. For null columns, populate them based on how likely you think they're be used, but err on the side of more space.

Obviously, save your script for later.

4. Run SP_SPACEUSED

SP_SpaceUsed displays how much data a table is using. It shows results for both the data, as well as the indexes (never forget the index space).

You can run it as simply as:

exec SP_SPACEUSED 'TableTest1'

Now you can get a unit-size per row. For example, if the table has 3000KB for data, and 1500KB for indexes, and you inserted a 100K rows, then the average size per row is: (3000KB + 1500KB) / 100,000. Then, multiple that by however many rows you expect.

This may seem like a lot of work, and there are certainly ways to theoretically predict it by plugging into a formula. My concern is that it's too easy for devs to miscalculate the formula (like forgetting the indexes, not accounting the initial table schema itself, or just all the extra steps)

5. Estimate the expected growth

Knowing the initial size is great, but you also must be prepared for growth. We can make educated guesses based on the driving factors of the table size (maybe new customers, a vendor data feed, or user activity), and we can then estimate the size based on historical data or the business's expectations. For example, if the table is based on new customers, and the sales team expects 10% growth, then prepare for 10% growth. Of if the table is based on a vendor data feed, and historically the feed has 13% new records every year, then prepare for 13% growth.

Depending on your company's SAN and DBA strategy, be prepared to have your initial estimate at least include enough space for the first year of growth.

6. Add a safety factor

There will be new columns, new lookup and helper tables, a burst of additional rows, maybe an extra index - something that increases the size. So, always add a safety factor.

7. Prepare for an archival strategy

Some data sources (such as verbose log records) are prone to become huge. Therefore, always have a plan for archival - even if it's that you can't archive (such as it's a transactional table and the business requires regular transactions on historical data). However, sometimes you get lucky; perhaps the business requirements say that based on the type of data, you only legally need to carry 4 years worth of data. Or, perhaps after the first 2 years, the data can be archived in a data warehouse, and then you don't worry about it anymore (this just passes the problem to someone else).

Summary

Here's a sample T-SQL script to create the table and index, insert data, and then call SP_SpaceUsed:

USE [MyTest]
GO

if exists (select 1 from sys.indexes where [name] = 'IX_TableTest1')
    drop index TableTest1.IX_TableTest1

if exists (select 1 from sys.tables where [name] = 'TableTest1')
    drop table TableTest1

--=========================================
--Custom SQL table
CREATE TABLE [dbo].[TableTest1](
    [SomeId] [int] IDENTITY(100000,1) NOT NULL,
    [phone] [bigint] NOT NULL,
    [SomeDate] [datetime] NOT NULL,
    [LastModDate] [datetime] NOT NULL
) ON [PRIMARY]

--Index
CREATE UNIQUE NONCLUSTERED INDEX [IX_TableTest1] ON [TableTest1]
(
    [SomeId] ASC,
    [phone] ASC
) ON [PRIMARY]
--=========================================


--do inserts

declare @max_rows int
select @max_rows = 1000

declare @i as int
select @i = 1

WHILE (@i <= @max_rows)
BEGIN
    --=============
    --Custom SQL Insert (note: use identity value for uniqueness)
    insert into TableTest1 (phone, SomeDate, LastModDate)
    select 6301112222, getDate(), getDate()
    --=============

    select @i = @i + 1

END

--Get sizes
exec SP_SPACEUSED 'TableTest1'

1. Wild guess - sometimes this seems like your only option, but most of the time there is ways to improve on the guess (base it on similar projects, or split it into components and estimate those individually).
2. What you think the boss wants to hear - This may seem like the easy way to initially win favor with the boss, but it will come back with a vengeance when the estimate drastically deviates from reality. Also, because the boss always wants to hear lower schedule times, this has a huge bias that will send you off course.
3. Base it on unrelated projects - Historical data is great, but don't compare apples to oranges. That a WinForm app took 3 months tells you almost nothing about how long an ASP.Net app will take.
4. Pick an arbitrary big number - During crunch time, it's easy to think that everything will magically be better "next week" or "next month" ("that will give us enough time to fix everything),  but then that time rolls around and the project is still behind schedule.

All of these are bad estimation methods because they miss the fundamental point - how long will the project really take to build in the real world? Wild guesses or the boss's wishes are not necessarily grounded in reality, so basing estimates on them is barking up the wrong tree.

I realize it's easy to say "how not to do something". I'd recommend Steve McConnell's book, Software Estimation: Demystifying the Black Art, for how to do a great job of estimating.

The books consists of 97 short essays, by various accomplished architects, each with a quick insight. It was a casual and fun read, the kind of book that's easy to sneak in a few pages between changing the kids diapers and fixing the house. It's got a lot of good points. I especially recall an essay about how "the database is your fortress" - GUI and Middle Tier apps change, but you will always have your database.

For hard-core architecture, I thought that Microsoft .NET: Architecting Applications for the Enterprise and Patterns of Enterprise Application Architecture were much more systematic and thorough. But overall, it was still a fun read.

• Infrastructure (Servers, SAN space, database access, network access)
• Design decisions
• Dealing with legacy code
• Handling outsourcing, insourcing, consultants
• Build-vs-buy
• Vendor evaluations/score card; integrate the vendor's product into your own
• Coordinate large-scale integration of many apps from different environments
• Coordination among multiple product life cycles
• Writing guidance docs
• Code reviews
• Occasional prototypes
• Configuration

On one hand, these types of tasks require technical knowledge in that you wouldn't expect a non-technical person to perform them. On the other hand, they don't seem in the same category as hands-on coding.

What do you think - can you be technical (or remain technical) without actually writing code?

Perhaps for SQL, I can see the benefit, so that when the DBA runs sp_helptext on a stored proc, they get a quick history (and databases aren't usually refactored like C# code). However, for middle tier code, putting revision history in comments seems like an unwise use of time.

LINK: Comments as version control
 

• Are scalable - they handle large loads and can be called many times.
• Have a retry strategy - for example, it tries pinging the external service three times before "failing".
• Have a failover strategy, like an active-passive machine cluster for maximum uptime, and a disaster recovery site.
• Send notifications.
• Handles invalid data (like states, zip codes, and numbers).
• Can integrate with other systems (perhaps providing web service wrappers, or command line APIs, or publicly accessible data repositories that other apps can modify) .
• Are deployable - "it works on my machine" absolutely does not cut it.
• Have Logging - this is especially useful for debugging in production, or measuring how many errors (and which types) are thrown.
• Have long-running process (hours, days, or even weeks) - not just a single thread in memory.
• Have async processes, which usually means concurrency and threading problems.
• Support multiple instances of the app running. You can open two copies of word, or run two MSBuild scripts at the same time.
• Handle product versioning.
• Care about the hardware it's running on (enough CPU and memory).
• Have a pluggable architecture - You may need to switch data providers (Oracle/SQL/Xml).
• Have external data sources (web services, ftp file dumps, external databases).
• Can scale out, such as adding more web servers, or splitting the database across multiple servers.
• Have security constraints (both hacking, and functional).
• Have process that are documented (not just for training, but also for legal auditing and compliance issues).

Much of this code isn't the fun, "glamorous" stuff. However, it's this kind of robustness that separates the "toys" from the enterprise workhorses.

See also: Enterprise Data Access, Enterprise Caching

Books have been written on caching, so this is just a quick brain dump based on my personal experiences. Also note that I refer to "the database" a lot because it's the main data dependency that most developers can relate to, but really, it could be anything (web service, external file, etc...).

Frontend and Backend Caching

Obvious follow-up question: "Would you double cache something, taking data from the backend cache and saving it to the fronend cache?" Sure, if it benefited your specific scenario. Ideally the backend cache is totally encapsulated anyway, so your frontend UI developer wouldn't even know if the data they're working with came from a cache or not.

What is a good candidate for caching?

Any data that:

• Takes a lot of time to create, either due to a remote hit (to a database or web service), or a large calculation time (like querying a million rows).

• Has a small final result - you query a million rows just to return a single value.

• Does not change - the problem with caching is stale data.

• Has minimal dependencies. If your object touches 10 tables for creation, then there's a much greater chance of it becoming stale. This is one benefit of loosely coupled (and then batched) objects, instead of spaghetti code.

• Has many reads, but very few writes. For example, system-level data (that everyone constantly requests) is good for caching, but employee-level data may not be.

• Is retrieved externally and requires high uptime - for example you make a web service call to get some value, you call the service again 5 minutes later, the service is down, and you really wish you had even a "stale" copy of that data.

The canonical example would be something like city-state dropdowns. Say it's initially a remote database hit to some "City/State" tables, it returns a small amount of data, it changes infrequently (The US has had 50 states for the last half-century), so it's read many times but not prone to stale data.

What is a bad candidate for caching? Pretty much, the opposite of the good criteria.

Pitfalls with caching

Merely creating the dictionary isn't the problem. The problem will be integrating it (seamlessly) into your data persistence layer, and then ensuring the cache doesn't become stale - especially across a distributed environment - and then making sure it actually improves your performance instead of degrading it.

Stale data is the bane of caching. There are at least a few ways to deal with stale data:

• Apply a time-out policy such that all data expires after N minutes, but that won't be acceptable for most scenarios.
• If your app is changing the data (such as updating a details page), then have the DAL method (that calls the update SQL) also ping the cache to clear the stale object. This requires that your application somehow keeps track of which objects depend on which piece of data. It works great with a domain model, which requires more design upfront, but can have huge payoffs.
• If someone else is directly changing the database, such as a DBA running ad-hoc SQL in production, then consider providing some admin console that lets them clear segments of the cache. For example, if the DBA did a mass-update of all salaries, then have the admin page allow you to flush all salary-related objects out of the cache. This requires some infrastructure for tagging each cached object (perhaps a master config file), and discipline from the DBA to check that admin page.
• Beware of "database cache dependencies" (ASP.Net 2.0?), that claim to let you apply triggers to the database table, and then automatically clear the appropriate cache items when a specific row/column is updated. I've personally never gotten this to successfully work, have heard lots of horror stories (especially when deploying it across a DMZ), and it forces the domain design into the database instead of the middle tier. Although I'm all ears to anyone who's had a success story here.

Some other things to keep in mind:

• Where should my cache tie in? Ideally, you'd want the backend cache abstracted via your dataAccess layer. This becomes very feasible with CodeGeneration. Whether data is pulled from cache or the live database is just a tuning option. Just like you don't want to put in-line SQL throughout your codebehind pages, you probably don't want to tightly-couple all your UI code to your cache provider. The frontend cache can be referenced in your UI, but again, be aware of too much plumbing code that "designs you into a corner".

• Only temporary - A cache is not a persistent data store, it is not merely a "mirror" to scale out your database, as you must account for the cache being cleared. A data access method should always have a means to recreate the cached data.

• Why use a remote cache? If both the cache and database servers both have remote hits, what's the difference? In its simplest form, this helps scale out the database (which is usually a bottleneck). Every hit on the cache is one less hit on the already-overloaded database. Even after the remote hit, the cache can have a much faster lookup time because while the cache stores a created object, the database may still need to query 1 million rows to collect the data.

• Control Panel - You'll want to provide an easy way to flush the entire cache, or even segments of the cache, without restarting any servers. It's also great moral support to have a statistics page showing how many thousand (million) database calls have been spared.

• Configuration - You'll want to provide a way to configure almost everything: the cache durations, what category of duration (short, medium, long), which objects get cached, and perhaps even turn off the entire cache for emergency troubleshooting. Caching is something that you want to tune based on actual production results. Ideally control of what gets cached is all abstracted to a few easy-to-manage config files. You do not want these config values hard-coded throughout your app.

• Beware of over-caching - If done the wrong way, caching can actually screw your performance. Say you cache something that is continually obsolete, so instead of just doing the normal database hit, you continually also need to do the extra cache query.

Good things to read

There's an endless list of info on caching. Here are some that could be useful.

• The Pragmatic Programmers have a whole book on caching with memcached! (I've personally used Memcached before, and liked it.)
• MSDN: Caching Architecture Guide for .NET Framework Applications
• Enterprise Library Caching Application Block

Yes, there's ton more that can be said about caching. Again, this is just a quick brain dump.

A lot of developers will try fixing this by modifying the connection string by appending "Connect Timeout=300". Normally this is easy because the connection string is stored in some config file.

However, it still usually fails. This is because there's a big difference between SqlConnection.ConnectionTimeout and SqlCommand.CommandTimeout.

If you're running a command, like a snippet of SQL or a stored proc, then your code needs to set the CommandTimeout. Something like so:

SqlConnection con = new SqlConnection(strDbCon);
SqlCommand cmd = con.CreateCommand();
cmd.CommandType = CommandType.Text;
cmd.CommandText = strText;
cmd.CommandTimeout = 10000; //no relation to con.ConnectionTimeout

Obviously, that's very minimalist code, but that's the general idea. For a robust data access layer, you'd make the timeout configurable.

Same thing with software projects and best practices. The project is like jumping out of a plane, and the best practices are like the parachute.  Sure, some "best practices" are just useless marketing buzzwords. However, others - like unit testing - are the real deal. And a developer or manager who rejects unit testing (for new .Net code in the middle tier) because they "don't have time" is like jumping out of the plane without putting on your parachute. You save a bit of time upfront, but when the project goes to production and "crashes" into reality, the maintenance costs and untested boundary cases will kill it.

"I don't have time" sounds much more noble and business-like than "I don't understand that idea", or "I just don't feel like doing something new." But within the (very common) context of new .Net class-library development - given that testing frameworks are free (NUnit, MSTest), and that any developer can at least write their tests locally - regardless of management support, and that in certain scenarios it's actually faster to develop code by writing unit tests (because it stubs out the context), the "I don't have time" reply to unit testing certain scenarios is misguided. Sure there are always exceptions, and reasons that good devs may not write unit tests. Testing databases, or legacy code, or the UI, or integration is a different beast. However, in general, testing public methods in a C# class library, without external data dependencies, should be as reasonable as putting a parachute on while jumping out of a plane.

This is a great event for those interested in C# 4.0.

• Do you allow for special characters, like dollar signs "$", percents "%", or financial negative numbers "( )"? [Seriously, try formatting a negative dollar amount in Excel]
• Do you allow commas?
• Do you allow decimals?
• Do you handle globalization? Some countries use the "," for a decimal point, and the "." for a comma (the inverse of what America does).
• Do you allow both "-" (for negative) and "+" for positive, or is the "+" just implied by default?
• Do you allow pre and post zeros: "001.00". Keep in mind that the zeros on the left are mathematically redundant, but the zeros on the right may count as "significant digits", and denote a degree of precision. The extra zeros in "1.25000" implies a much more precise number than just "1.25". However, most numeric types in .programming languages will truncate the trailing zeros.
• Do you allow exponents for large numbers, like "1E+12" (that's 1 with 12 zeros after it: "1,000,000,000,000")?
• Do you allow abbreviations, like "10M" for "10,000,000"? (FWIW, I'm not sure of any app that actually does this.)
• Do you have a clearly defined range? For example, a smallint (Int16) stores much smaller numbers than a long (Int64). Most business values can be stored with a double. However, if you're extending a legacy system that only provides a byte or smallint, but it expects occasionally huge values, then you could get into trouble.
• Is your validation lenient? For example, if the user enters multiple commas "23,5,6" - do you simply remove all the commas and have "2346", or is that an error?
• How do you handle nulls? Do you use a sentinel value (like Int32.MinValue), or use the new Nullable data types, or something else? If using a nullable value, does it "convert up" - i.e. Int32.MinValue is null if the type if Int32, but it would be a valid non-null value if the type was Int64. If you have multiple systems reading the same value, and one system uses Int32, and the other uses Int64, will your system still work (this can happen when you start doing things like letting the user dynamically create their own pages or reports).
• Not to be smart-alecky about it, but I'd just assume that unless otherwise told, this accepts base-10 inputs. I.e. "A5" and "FF" are invalid values - although there may be apps where that is legit (like specify a color in HTML for some blog or content-management software).

Also, does your system handle transformations? That's where you collect the input, store the raw data as something else, and display a newly formatted value. Perhaps the most common example is with percents. A user may type "50" into a textbox (with the "%" in the label next to it), you may save it in the database as "0.5", and you may format it back as "50%" on a report. Or a user may enter "+0" in a textbox, and you simple store and display it as "0".

So, some numerical inputs to test with (besides the obvious invalid numeric inputs, like "xyz") :

• -23,456.000
• -23.456,00    //Globalization
• $45,345.00
• $(4.00)
• The max/min values for each of the numeric types.
• Fill the entire textbox with 9's.
• 000000000
• +23
• 23     //try the simple case
• -0
• +0
• 1.25E+12
• 1.25 E + 12    //note the spaces

The things to look for are (1) will these prompt validation messages, (2) how will each value be stored in the database, and (3) how will the value be re-formatted when it's displayed.

Perhaps the best standard is to see how Excel handles it, as Excel is probably the most famous application in the world that handles numbers.