When Your Model Doesn't Work (for Rookies)

I think hours and hours rolling on this roller coaster had made me a more strategic woman although sometimes bitter or too overenthusiastic.

With certain software, it isn't always "easy". I mean, I get it. We got to put in the years and learn the shit. Usually, the advice is that your model will work faster and better with experience. Like most millennial, I am impatient and oh... the struggles... *cue over-dramatic music*

When you bug the people you know about why your model is not working, usually, people try to help. Deep down, you know this.

Truly no one cares. No seriously, not even the paid support care. The less you contact them, the better... for them. Sometimes, I've waited 16 hours to get a question echoed back at me or literally two words back. Granted, the support teams are bombarded with many questions daily, so my appreciation for them is as much as I appreciate my siblings. Just sometimes, they don't always solve your problems.

In the past 2.5 years in doing more of energy modeling, I've learned that there are strategic approaches toward the practice through organization thinking. It has helped my sanity a bit and here are the top 6 things I would tell my 2.5 years younger self.

1. Control Your Changes

Try retracing your steps. Every time you made a change in something, write it down. Sometimes, I even take the screenshot of the changes and make a note. The best way I've learned in tracking my changes are through making a list in One Note or whatever note taking method you take. Try to run the sim and see the effect of that change when possible. If you don't know what the effect of a change you're going to make, the less control you have over your model.

Sometimes, this means a vigorous version control will save a lot of time. The tough part is always remembering what changes you made. Tracking your inputs in an organized manner also help your QC-er. If you found this assembly in this detail on page Ax.xx, it will save time for the QC person and for your own documentation at the beginning of your professional career.

With energy model, BuildSimHub offers this version control like a github for your model. Yeah yeah yeah, it's too techy for you. I understand. I hated git at first too. Keep it simple and use a notepad. I usually don't do this until I'm in "test-running-QA/QC" mode or running multiple options later in life. It's cool when parametric abilities are automated within the software, but I feel like it's still a work in progress for most of them or I'm too stupid to use it.

2. Think like a Programmer (There's an excellent talk by Michael O'Keef and Peter Ellis at IBPSA conference in 2016 on this. Here's the paper version.)

How does the programmer learn a language and get better at it?

Read the documentation.

"So what? I'm not a programmer." Me neither, but a programmer made this software with his/her mind. Blaming the software isn't going to make your model work.

I get it. It's a dull giant book like the specs. You feel this giant resistance telling you , "can I just google this?" Sometimes when the answer is yes. If google doesn't solve your problem in 5 minutes or less, just read the damn manual. Someone spent a million hours on writing those 3,000 pages. When the user manual sucks, check the Engineering References. When the engineering references made no sense, consult the internet. If the best answer(s) cite the damn thing again, track down the developer. DEVELOPER. Not support. Not sales. The developer. So you can ask them this.

All kidding aside, when you run into a wall and talk to a developer. A systematic way to do this is through report issues or requests from git. Not all software have this avenue. Sometimes it's back and forth through support emails.

3. Walk away from the model.

When you stare at something for too long, you are going to make the same mistakes. When you are doing the same things over and over again while expecting a different result, it's going to make you insane. Believe me, insanity is no fun.

When you feel that blood boils, walk away from the model, the project, life, whatever. Come back to it when you almost forgot about it and sometimes you can see the glaring problem right away.

If it makes you feel better, try to post the questions you have somewhere (support, forum, coworker, that energy modeler in the other company) so there's maybe some hope someone else can enlighten you. Well, at least it makes me feel better when someone else knows I'm dying inside.

4. Isolate the problem

I made a lot of mistakes as a rookie. One of them was making multiple changes and trying to recall which change made the most impact. I barely remember all the changes I made because Tip #1 didn't kick in yet. There are other times when I've made all the right inputs, but have not understand the software intrinsic behaviors. My output results were changing drastically due to controllers from default input to the slightest changes.

Sometimes, it felt like the monster inside simulation model has a mind of its own.

With software that has longer run time and/or with a new building type that you're not used to troubleshoot, this is extremely painful to isolate the problem.

What really helps is dumb-ing down your model. If you have a 24 stories building, cut it down to one floor. Test run the crap out of that floor on the HVAC systems and make sure its outputs are reasonable and expected. Then apply this to your official model. This ultimately saves time and - more importantly - save your sanity.

In case where you can assign ideal HVAC system (dummy autosized HVAC or ideal loads), try to run the model with that box checked. It validates whether you got the step envelope to internal gain mostly right. When you do test run, I would always pick 1 winter and 1 summer month. Depend on which energy savings measure I know that this building will have, I usually have a good idea how much savings should be for each end use. HVAC end usages' savings are the most tricky since it's usually interactive with other measures and end usages.

When you solve through your problems, take up one task at a time. When you are making multiple changes, those changes should have independent outcome. If there are some small and predictable interactive effects, it's fine. If it's extremely codependent, you should make one changes at a time.

5. Treating Errors by Severity

It's important to have a good attitude when you encounter problems or 2342343 errors in your Energyplus error log. It's okay. It's alright. If you're lucky, you have an error log. Nowadays, my errors log composes of the problems I see from the output results compared the software generated one. Looking at the results by pretending you're QC-ing someone else's model will make you less bias and actually thinking through whether the results make sense.

I try to take tabulate my errors (in my brain) by severity. If the most important errors (usually geometry related) are causing the simulation to be confused. Often with LEED modeling, there's coordination and requirement of consistency between credits that make this process sometimes painful. When you fix the major problems, it might incur changes to your systems and resolve other errors on its own.

Let's Thanos those other half errors away.

Sometimes, you have to think about what errors and what changes will make the most impact to your model and affect the results (whether that's % saving or % cost savings) most drastically. If there are errors that contribute to little to no change, they don't matter. Sure, it's not 0.045, but 0.058 U-value, but how does it impact the energy results.

6. Understanding What Your Work is Being Used For

With over thousands of variable for you to enter and keep track up (and maintaining them with changes in design), it's hard and extremely time-consuming to be perfect. Of course, we always strive for it. Realistically, we aim to be accurate and as precise as possible. It is better than being somewhat precise and completely inaccurate, leading others to think energy modeling is simply overpriced useless annoyance and energy modelers don't have a grasp of performance reality.

At the end of the day, the precision and accuracy of the model depends on the context of which it is being used for. For example, are you using the model as a relative comparison tool? In that case, perhaps precision is more important than accuracy because you're only interested in the incremental results . If you are to create a super duper precised and accurate model and spends 3x the amount of hours you did to come to the same conclusion - my friend, you achieved your goals the first go round. It seemed obvious, but for a rookie, it was a long lesson learned.

Do you agree with these approaches? Did you went through any other rookie mistakes? What better advice would you give to your younger professional self?