Setup, run, and get HFSS simulation results in one click !

Herding Cats; Creativity and Discipline in Engineering Development

I was working on a team designing a large computing system.  It was a long-term project with lots of funding so we had time to explore many ideas.  The group had many academic/professorial type engineers.  There were no urgent deadlines.  After a while, we were working on lots of interesting side projects that had little to do with completing the task.  One engineer was working on chemical surface treatments to reduce PCB surface roughness, rather than designing a board for the project.  It was fun and a good learning experience, but not directly relevant to completing the project.  We needed more focus and discipline to make steady progress.

I've also worked on a team at the other end of the spectrum.  The project was so structured and regimented that we successfully completed a product, on time, that nobody wanted.  The product was dull, expensive and had mediocre performance at best.  The team needed more creativity and experimentation.

Like most things in life, moderation and diversity is best.  The strongest engineering teams include people with a diversity of styles and skills.  The best teams need a combination of order and chaos.  Flexibility, creativity, uncertainty and experiments lead to new ideas and discoveries.  The most effective teams I've been on, have developed spontaneous order.  Everybody has the right motivation and the right mix of skills and leadership that people can see what needs to be done next and just do it.  Everybody does their job and looks for opportunities to make improvements and do something valuable.
On the other hand, some amount discipline and enforcement is needed occasionally to encourage team members to stay focused on the goals of the team.  There are certain times where what's best for the individual is not what's best for the team.  It's good to avoid these situations as much as possible, but they will happen occasionally.  It's in these moments, that a little bit of discipline makes a huge difference.

For instance, a project task may require 1000 measurements.  It's tedious and time-consuming and most engineers do not find it fun.  Also, it's a thankless job, with little reward and low prestige and fame.  An engineer is unlikely to get a raise or promotion for completing repeated measurements.  However, those tedious measurements are critical to build an accurate dataset, in order to make the right decisions for the project.

What's a good way to have structure and focus, while encouraging creativity and experimentation?  What's a good way to give people flexibility and freedom, while staying on schedule and making steady progress?  How do we have good teamwork and encourage people occasionally to make individual sacrifices in order to provide a big benefit to the team?

There are many pieces to building good teamwork.  These include focusing on goals and having a clear vision.  It's also important to address obstacles to teamwork.  One of the biggest obstacles is knowing when to quit.  When an idea doesn't work, it's difficult for people to move on.  We really want our ideas to work!  If there's a huge penalty to telling people that something is not working, then people will cover it up or ask for more time.  Engineers can make it sound like it's almost working, not ask for help.  It's better to get help, try really hard multiple times, then shut it down if it's not working and move on.  Another big obstacle is the credit-grab.  People want to position themselves so that they get the credit if the project is successful, but somebody else gets the blame if it's not.  It works well to give the team credit for success and reward people for finding problems, rather than penalizing people for problems.

I've had the most success in engineering teams that maintained focus with clear goals and vision, lots of flexibility, and a little bit of peer pressure and minor enforcement when things get off track.  Here are the highlights of what has worked:

GOALS, VISION
  > one big idea, measure everything as to how much progress is made toward the vision

Keep focus on Solving the problem statement, delivering a solution for a customer, making a product

Give people 20%
   > 20% of time can be spent on your own ideas, basically whatever you want

Peer pressure and nudges in the right direction
   > regular meetings with status updates, focus on vision
   > give praise for teamwork and results

Share the load
   > the most painful and tedious tasks can be shared and supported by the team
   > have a measurement day, everybody goes into the lab for 3 hours and gets the 1000 measurements done
   > have a documentation day, bring in coffee and donuts in the morning, get into a conference, setup your laptops and work together to get the documentation done

Winning makes everything better
   >  Just like in sports, winning helps smooth over any technical or relational challenges
   >  Focus on wins, success and accomplishments to provide motivation and energy to encourage teammates to work hard and make sacrifices for the team

Encouragement and praise for team results, trying something new and good attitudes
   > In order to counteract the habit of hiding bad results, heavily reward when something is proven not to work.  We have a conclusion and can move on, that's great !
   > Thank people for sharing and helping each other. For instance: "Lin, thank you for helping the analysis team to get the channel results."
   > Call out team results that make progress toward the goal. For instance: "Great job on the analysis results!  That gets us closer to completing a successful design."

Discourage credit-taking and blaming others
   > If somebody says "The mechanical engineer didn't get the drawing done, so the product is late,"  respond with facts and focus on goals and improvement: "Let's not assign blame.  The mechanical drawing was delayed by 3 days, so that delayed the product.  What do we need to do now to move forward with the product?  What can each of us do in the future to speed up the process?"

People can form and disband small teams (3-4 people) whenever they want, in order to accomplish specific tasks

Develop appreciation for each person's skills, say "good job" and mean it and appreciate it.

Put people in roles where their style and personality fits
   > People who are detail-oriented and good at following all the steps in a process can do measurement and methodical analysis
   > People who are creative and thrive on doing new things can do experiments and try creative solutions
   > People who like talking, interacting with people and selling an idea or vision can get drive schedules and gather input from everybody on the team to guide decisions
HAVE FUN !

Valuable Documentation; Do More With Less

Valuable Documentation; Do More With Less

Does anybody read the instruction manual?

The answer is "No," almost no one reads the manual, especially new users.  (I don't read the manual either.)

Still, good documentation is highly valuable to both new users and advanced users of a tool.  (tool = software or equipment or machine, etc.)  The best documentation is easy for the writer to write and update, provides simple instructions for new users, and gives access to detailed explanations for advanced users.  How do developers help users learn tools, and focus our documentation effort on what's useful for advanced users?

In the early years of my engineering career, I wrote a tool to calculate impedance.  It was a spreadsheet-based calculator that collected, in one place, characteristic impedance formulas for co-ax, microstrip, stripline and a few other structures.  I shared it with some other engineers and there were a lot of questions about it.  So I spent many hours documenting it, using screenshots and explanations, detailing every equation used and writing about how to use the tool.  The documentation effort consumed many times more hours than the writing tool. The tool took about 8 hours to build, while the documentation took about 40 hours.  After sending out the documentation to the users, it was clear that the documentation was a failure.  There were the same number of questions and same amount of confusion as when I had zero documentation.  The documentation Was not a good use of my time.

Basically, people asked two types of questions:

TYPE-1:  As a new user, how do I use the tool quickly to get results?

TYPE-2:  What exactly is the tool doing and are the results correct?

I thought I had addressed question 1, by explaining the entries in the spreadsheet and showing some screenshots.  The problem was that the information was methodical, but disorganized and didn't have a story or workflow.  After rewriting the documentation to highlight workflows and recipes for the most common tasks, the number of questions from new users dropped by about 75%.

For question 2, I thought that showing the formulas and calculations was sufficient.  I learned that engineers like to see comparison to known reference data (usually measurements) and like to see acceptance from other experts.  For the closed-form equations, I added references to the textbooks and papers where they were published.  I also added one example of a highly accurate simulation, compared to the results of the tool. 

(Side note:  When doing a tool accuracy study, I recommend doing one and only one example.  Doing more than one has vastly diminishing returns, because accuracy studies are so time consuming.  References, comparisons and one example is sufficient for almost all users.  For the one or two users where that's not enough, a one-on-one discussion works well.)

After updating the documentation, the number of type-2 questions dropped by about two-thirds. 

Here are some guidelines for spending a minimal amount of time and generating useful documentation.
There are three and a half purposes for documenting something:

1. Help new users start using the tool effectively.

2. For advanced users, provide detailed explanation of what the tool is doing and how it works.  Also advanced users want to know how to do some exact task that they can't figure out and how to do complicated, high-value, less-common tasks.

3. Share information between developers so they can improve the tool and update the documentation effectively.

3.5  When the manager says "Is this thing documented?"  Allow the engineers to reply "Yes, of course!"

There are many typical problems with documentation.  Here are some negative scenarios to avoid:

- People spend hundreds of engineering hours writing a 200-page instruction manual, then nobody reads it.  Users contact customer support with every question.

- New users look for information on how to use the tool, and either can't find the documentation, or don't understand it.  Perhaps they start reading it and give up because it's too long / annoying to read / is too complicated.

- Advanced users want to know how something works or want to do some advanced tasks.  They read the documentation and can't find the explanation that they need and can't figure out how to do the task they're trying to do.

For new users, the best documentation is a well-designed tool.  Users can start using it intuitively; the default settings do something useful and informative.  Also, have a step-by-step tutorial that doubles as the test plan.  Do this in detail, including example inputs and expected outputs.  You can give this to anybody and they can test your thing.  It also helps you, the developer, to remember every test step.

For advanced users, two main things are needed:

1. Searchable index of all features of the thing so users can find and use that one powerful feature they need and can't quite figure out.

2. Explanations of the unique, novel features of the thing.  Basically, try to anticipate questions and explanations that a highly experienced engineer would want to know.  For instance, a tool that plots s-parameter results doesn't need much explanation for plotting magnitude and phase.  It just reads values directly from the s-parameter and plots them.  On the other hand, for the within pair differential skew results, a detailed explanation is needed.  Advanced users need to know exactly how skew is calculated in order to understand the results.

It's often good to avoid using pictures.  A picture is worth a thousand words, but costs a hundred revisions.  Every time something changes, the documentation needs to change. Changing a picture takes 100x longer than editing a few lines of text.  Plus, text is a universal format; it works on computers, mobile devices, email, etc.  Many pictures in documentation are outdated and don't match the tool anyway.  Thus, I recommend avoiding pictures in documentation, it just takes way to long to create and maintain the pictures.  Using 100% text is so much faster to document than anything with pictures.

GUIDELINES FOR DOCUMENTATION TEXT

1. Write and update the documentation text so it matches exactly with thing.  For instance, with a save button in a tool, use the same upper and lower case and same words in the documentations as in the tool.  If the button reads "Save File" then the documentation should read "Save File" not "save" or "SAVE FILE."  Use consistent formatting, such as quotes or brackets or bold font, to highlight that the documentation is calling out a label verbatim from the tool.  Whenever a tool label changes, update the documentation so that it matches.

2. Draw simple text pictures to help the user understand.  For instance, tell the user what happens when the "Save File" button is pressed.  Draw an asci text picture of the button:

__________
|                   |
| Save File   | <--- Press "Save File" button and results are saved to a .s4p s-parameter file.
|_________|

Here's an example of how to make useful documentation without too much pain and time. S4PTOOL is a general tool for viewing, plotting and making reports from s4p s-parameter files.

SNAPSHOT OF S4PTOOL

The documentation text window pops up when the user presses the "?" button next to the 

“S4PTOOL” title.

>>>>>>>>>>   S4PTOOL DOCUMENTATION   <<<<<<<<<<

What this tool does:

   This tool is a general purpose .s4p viewer tool.

   This tool takes one or more .s4p files as inputs, and generates 4 plots on the screen, based on the current plot settings.

   Plot setting include differential or single-ended, magnitude or phase, and port configuration.

   It makes a report using powerpoint slides.

How to read this help file:

   People who learn by experimenting can stop reading now and just try the tool.

      The tool is designed to be intuitive and simple.  Select File(s) -> Plot Graphs and try a few settings.

   New users can go through the "S4PTOOL Step-by-Step Example" to learn how to do the most common tasks in the tool.

   NOTES FOR ADVANCED USERS section has a few explanations and notes about advanced features.

   COMMAND LIST is a description of commands in the tool.  This is especially useful when searching for specific features.

>>>>>>>>>>   STEP-BY-STEP EXAMPLE   <<<<<<<<<<

*** EXAMPLE FILES ARE FROM THE TOOLSEXAMPLES DIRECTORY ***

https://....toolswebsite.com/Tools/Examples

Read version at the top of the window     :  Is it the latest version 1.0?

Click "Select File(s)" -> Choose Data1\*.s4p (all .s4p files)     : Observe the passivity warning, which is very common with measured files.

   The message window should show "Files loaded!" when the loading is complete.

   HINTS AND TIPS:  Rt-click on the Current Folder entry of the select files window to access a history of recent directories.

Click "Plot Graphs"     : Observe viewer plots

Enter "1" in "Min. Freq..", Enter "20" in "Max. Freq.." -> Click "Update Range"

Choose "Single-ended" -> click "Plot Graphs"    : Observe viewer plots

Choose "Phase" -> click "Plot Graphs"     : Observe viewer plots

Choose "33,34,43,44" in Matrix Section -> Choose "Differential" -> Choose "Magnitude" -> Click Plot Graphs

      :   Observe plot results (Typically this is common mode performance)

Click Autoscale checkbox to turn off automatic scaling and enable manual plot axis scaling.

Enter "-30" in upper left "Min Y"   :  Observe viewer plots  (This changes the y-axis for the upper left and lower right plots, typically 11,22 reflections.)

Enter "-5" in lower left "Max Y"    :  Observe viewer plots  (This changes the y-axis for the lower left and upper right plots, typically 21,12 thrus.)

Observe "File Port Impedance:" label, this shows the .s4p port impedance

Click on lines in each plot to remove them and focus on the remaining lines   : Observe how the plot changes

Click "Show Legend"     : Window pops up showing plot legend; the file related to each plot line color.

Click "Create Report"   : Powerpoint slides are generated with one slide for each of the 4 plots

      NOTE:  Observe slides and check that plots and legend are as expected.

      Save slides to desired filename.

>>>>>>>>>>   NOTES FOR ADVANCED USERS:   <<<<<<<<<<

> The tool plots unwrapped phase, in radians, by taking the unwrapped angle of the complex values in the s-parameter entries.

> For magnitude, the tool plots dB = 20*log10(abs(complex value)) in each s-parameter entry.

> If the user loads .s2p files, the tool builds differential results by copying the .s2p file to ports 3 and 4.

> By clicking on lines in the plot, the user can remove lines and focus in on the remaining results.

>>>>>>>>>>   COMMAND LIST:   <<<<<<<<<<

AutoScale checkbox

   On (default) allows the plot to autoscale the y-axis to the values in the lines plotted.

   Off uses the manual min, max values entered by the user, to define the y-axis range. 

      The upper min, max entries define the upper left and lower right plot ranges. 

      The lower min, max entries define the lower left and upper right plot ranges.

Create Report

   Press the Create Report button to create a powerpoint report from the current plots. 

   The powerpoint will have a title slide, a legend slide showing the file names, plus one side for each of the 4 plots.

   After the slides automatically open, the user must save the powerpoint file to the desired filename.

Error Message: "At least one file violates passivity. Check before continuing."

   If the energy from all ports, at any frequency, is greater than 1, this error pops up.

   This is common for measured low frequency points, such as 10MHz.

   It is the user's responsibility to interpret the plot results carefully.

. . . OTHER COMMANDS

Alphabetical, searchable list of commands, with a few sentences of description for each command.

. . . OTHER COMMANDS

<Question Mark Picture> [?] = Help button;

   Click on it to display help text.  If you are reading this, you probably already clicked the help button.

Select File(s) button

________________________________________

|                                      |

|         Select File(s)               | <--- Press "Select File(s)”

|______________________________________|

   Click this GREEN button and a window opens to choose the file(s) for plotting.  The default File Filter is “*.s*p” to find all s-parameter files.

      The tool supports .s4p and .s2p files. 

   The user may change the filter to *.s4p to choose only s4p files.

   There's a regular expression search, “Reg. Exp. Filter,” to filter file names further.  A simple filter of “test” will search all filenames case-insensitively with the string “test” anywhere in the name. There are multiple ways to choose a directory.

      1. Type in a directory in the Current Folder entry.

      2. Navigate from the current directory to another one using the directory pulldown, or the up directory arrow,

         or going down a directory by choosing a directory under the current folder.

      3. Rt-click in the Current Folder entry to choose from the recent history of directories.

   There are multiple ways to choose a file:

      1. Double-click on a filename.

      2. Left-click on a filename, then click the Add-> button.

      3. Shift-click and/or control-click to choose multiple files, then click the Add-> button.

   After choosing the desired files, click Done.

Show Legend

   Press the Show Legend button to pop up a window that shows the files used and the plot colors and styles for each file.

>>>>>>>>>>>>>>>>>>>>

The documentation above took about 2 hours to write, and takes about 10 minutes to update when the tool is revised.

So, the next time you have a tool or need to write documentation, try these guidelines.   Use all text, have step-by-step instructions for new users and for testing, and have some references for advanced users.

It only takes a few hours to write and it's easy to edit.  You might even use the documentation yourself!

Get Things Done Better; Pound the Rock, Find a Sledgehammer, Build a Howitzer

The stonecutter's credo of "pound the rock" is a popular theme in business and in sports.

"I go and look at a stonecutter hammering away at his rock perhaps a hundred times without as much as a crack showing in it. Yet at the hundred and first blow it will split in two, and I know it was not that blow that did it, but all that had gone before."
-Jacob Riis

It makes a lot of sense.  When working at something, follow a good process and don't give up, even when progress seems slow.  By persevering and trusting the process, the job will get done and be done well.
I have found this credo to be useful and true, so I've added to it.  My philosophy for getting things done is:

"POUND THE ROCK, FIND A SLEDGEHAMMER, BUILD A HOWITZER"

"Pound the rock" means following the process, and sticking to it.  For instance, in an engineering project, it means going through all the experiments planned and looking at all the results.   Sticking to the plan requires a good, clear plan, and a buy-in from the team.  

POUND THE ROCK

"Find a sledgehammer" means that while pounding the rock, think about how to get it done a little faster, a little better.  Improvements of 10% to 50% can be found by speeding up the slowest step or reducing the number of steps needed or automating a few steps.

FIND A SLEDGEHAMMER

"Build a Howitzer" means developing a method where the task is done in one or a few steps.  Blast that rock into tiny pieces in one, single step!  The user must have a clear vision of how to do the task and all the typical inputs, settings and outputs.  This requires changes outside of the specific task.  Inputs must change and be standardized to make the task simple and repeatable.  The task itself must be automated and set up to handle the most common scenarios.  The outputs must be well-defined and ready to pass to the next task with no edits.

BUILD A HOWITZER

As a signal integrity engineer, I'm constantly running simulations with field solvers.  Here's how the process looks in a specific scenario.

"Pound the rock" Simulation Method
> Make a new simulation file
> Load all the physical objects, typically a .STP file from the mechanical engineer
> Setup the materials for each object
> Setup the energy sources (aka ports)
> Setup the solve
> Run the solve, typically on a big compute array
> Extract the results, typically s-parameter files
> Final output = s-parameter file -> Goes to next task of analyzing the results
TOTAL STEPS: ~50 mouse clicks and a few entries typed in
TOTAL ENGINEERING TIME: ~30 minutes

"Find the Sledgehammer" Simulation Method
> Run a script/macro to make a new simulation file and load the .STP file
> Setup the materials for each object
> Setup the energy sources (aka ports)
> Run a script/macro to setup the solve
> Run the solve, typically on a big compute array, and automatically export s-parameter files when complete
> Final output = s-parameter file -> Goes to next task of analyzing the results
TOTAL STEPS: ~20 mouse clicks and a few entries typed in
TOTAL ENGINEERING TIME: ~10 minutes

"Build a Howitzer" Simulation Method
> Run a tool that takes a .STP file, and a settings file, press GO !
Makes a new simulation file and load the .STP file
Sets up materials based on the settings and lookup table
Sets up the energy sources (aka ports)
Puts in the solve settings
Queues the solve on the compute array
Exports s-parameter file with a clear, descriptive filename, and a detailed, standardized header file
Runs continuity, causality and reciprocity checkers on the data to verify good data
Puts the s-parameter file in a specified location, ready for the analysis task

FINISH THE JOB IN ONE STEP

> This method requires some infrastructure and standards to work well, especially for the input mechanical geometry file
3D geometry file must follow a naming convention such that the tool can match objects to materials
mechanical object to material lookup must be built; such as any object MAT12345_sigA2.... is assigned to Copper
mechanical object names must be labeled to indicate signal and ground metal; such as MAT12345_sigA2... = signal, MAT12345_gndC1... = ground
tool must be able to both read the settings file and implement those settings in the solver;
For instance: read start frequency and stop frequency from a text file, then run a script to setup those frequencies in the solver
TOTAL STEPS: 1 mouse click and 2 filenames
TOTAL ENGINEERING TIME: 1 minute

TIME SAVINGS CHART

What howitzers do you need to build?
What would it take?  What standardization?  What changes are needed for how the task is done?  What changes are needed before and after the task?
It's not easy to build the howitzer, but there's always a way to do it.  I encourage everyone to think about it and start building one!

Improve Productivity with RULES OF THE TOOLS

Webster's dictionary defines "TOOL" as "something used in performing an operation." Tools make tasks easier and faster or even make work possible when it would otherwise not be.  Using the right tool for the job makes tasks much more productive and less frustrating. Many repeated tasks, such as saving data and making reports, are a great fit for a computer tool to do the work.  If you have a good tool, learn it, master it and use it.  If you don't have one, then write your own !

RULES OF THE TOOLS

- Tool structure has 3 pieces
        Inputs: 
        1. Go into the engine for every output and can change                                constantly.       
            These are like parts in an assembly line.
        2. Settings:
                  Tell the engine what to do for each instance.  
            These change regularly, but not constantly.
        3.  Engine:
             Takes the inputs, reads the settings, does the work and                             produces the outputs.
             This can be updated to improve features or speed.


- Tool operation has no more than 3 steps
1. Read the settings
2. Put in the inputs
3. GO !  (Get the outputs)

- Use what you already have
  > Many software programs already have some scripting and macro features.  Use these features to make a tool
  > Microsoft office tools (word, excel, powerpoint...) have macro features.  Lab equipment has control commands.  Simulation software, such as HFSS, ADS, and CST, have scripting commands.
  > Matlab, Perl, and python are general software tools that can be used to make almost any other tool.
  > Think big!  Make a script/macro that completes typical tasks in a single click

- Must do something useful in the default condition
> new users just press go and something happens
> choose the most common usage for the default settings

- Has production mode and engineering mode
> production = fast, easy, 3 or 4 settings, then go
> production mode produces outputs which are ready to publish with no edits
> engineering = lots of optional settings, batch mode, can do almost everything the user might want
> optimize for the most common uses, handle the exceptions
> always build-in features for batch mode and command-line only usage

- Self-documenting, no instructions needed
> intuitive, like iphone/ipad
> context help pop-ups for new users, like build-and-wait smartphone games
> the tool tries to guess right, on the first try, about what the user wants, using contextual clues

- Expandable to add useful features without a major rewrite
> modular, abstraction layers
> easy for users to write their own code and enhance the tool
> built for crowdsourcing / open sourcing / improvement suggestions from users

- Automatically check as much as possible as quickly as possible
> use quick, automatic checkers to find problems with inputs and settings, before producing an output
> check any user inputs for typos and entries that don't make sense
> check measurements to verify valid results and find common problems before saving data
> work to make the checks very fast, on the order of milliseconds;  checkers that take as much time as the measurement, are not worth much

- Setup for continuous improvement; faster, better, cheaper
> always put information in a log file after every output, including date and time, operator and any other useful information
> always have metrics to measure progress and continuously monitor them through the log file
> make it like production, count production units, s4p's, reports, plots, designs, etc // continuously monitor speed, accuracy, etc.
> identify the slow steps in the process and figure out how to speed them up
> identify features that will significantly improve the engine accuracy, ease-of-use or add a new feature

- Build it in phases
> Phase 1: do something simple, useful, save 25% of repeated work
> Phase 2: add flexibility, speed, automation, save 50% of repeated work
> Phase 3: more speed, more features, has a way of doing 90% of user's tasks

Here's an EXAMPLE of using these rules to build a tool.

TOOL SNAPSHOT:  HERE'S WHAT IT LOOKS LIKE

I worked in a lab that measured cables.  Each was measured in a range of frequencies on the network analyzer and in the time-domain on the TDR/Scope.  The frequency data was stored in s-parameter files the time-domain waveforms were stored in CSV files.  In order to produce a production report for each cable, the data was cut and pasted, and screenshots were taken.  Then all the results were manually combined into a report.  All the files were saved manually; click-click-click on the mouse to get the equipment to save a file.  Then the user had to type in the name of the file.  For an 8-pair cable bundle, this required 8 s4p files, 8 CSV files and 16 screenshots.  In the worst case, it took 1.5 hours to measure the 8 pairs and make a report.  

I led a team to improve the process and make a tool that controlled the equipment, saved the data and generated a report.  Now the 8 pair measurements take 8 minutes total to save results and generate a report.  Also, the files have a clear directory structure and filename convention so they can be located and identified easily.  This makes it easy to make another tool to scan thousands or millions of files and extract useful data.

LAB TOOL DETAILS:
1. Inputs 
device under test
part number
unique tracking info (serial number, batch code etc)

2. Settings
measurement setup
frequency range
pass/fail specifications

3. Engine
runs checkers to verify that the measurement is ready and valid
performs a VNA frequency sweep
saves the data using a specific directory and filename
generates a report showing device information, pass/fail results, and plots

LAB TOOL WORKFLOW:
> CLEAR the entries from the previous measurement, in order to reduce the chance of mis-labeled data
> Enter part number and unique tracking info
> Press DATA button, which runs checkers and saves files
> Press REPORT button, which takes the saved data and generates a report
DONE !

EXAMPLE CODE FOR CHECKING THE INPUTS:
Check the inputs, make sure they make sense before spending time on data or files or analysis.

******* Excel Macro **********
pn = Range("B4") ' part number entry
rev = Range("B5") ' revision entry
Label = Range("B12") ' label entry
dataclr = Range("CZ1") ' load dataclear flag

If dataclr = 0 Then  ' check that old information is cleared before saving new data
  MsgBox ("Users must << CLEAR PRODUCT INFO >> before running this command.")
  End  ' if data is not clear, stop the tool immediately
End If

' Error checking, Make sure all the spreadsheet entries are filled
If (StrComp(Range("B4"), "") = 0) Or (StrComp(Range("B4"), " ") = 0) Or (Len(Range("B4")) < 10) Then
  MsgBox ("Please check the  << PART NUMBER >>  entry.")
  flag = 1
End If

If (StrComp(Range("B5"), "") = 0) Or (StrComp(Range("B5"), " ") = 0) Then
  MsgBox ("Please check the  << REVISION >>  entry.")
  flag = 1
End If

If flag Then
  MsgBox ("Some information is missing.  Please check the product number, lot, batch ... entries and re-start the macro.")
  End  ' if the labels don't follow the standard, stop the tool immediately
End If

In order to speed up and simplify the process, the part number points to a look-up table, in a spreadsheet, that loads in all the settings and specifications.  The settings and specifications are updated occasionally for new revisions or to adjust the frequency range of the measurements.  It takes only a few seconds to edit or create a spec.

EXAMPLE CODE TO SETUP THE NETWORK ANALYZER:
******* Excel Macro **********
Dim pna
Dim cset
Dim calsets
Dim myChan
Dim TRL_cal_ID As String
Dim E_cal_ID As String
Dim Filename1 As String
Dim Filename2 As String

Sheets("Test GUI").Select
E_cal_ID = Range("B13") ' get the desired calibration ID from the chart
MsgBox ("ECAL SET:  " + E_cal_ID), OK

'create the Network Analyzer object
Set pna = CreateObject("AgilentPNA835x.Application")
pna.Preset 'clear and reset the equipment to a known state

Set calsets = pna.GetCalManager.calsets ' get the calset collection
Set chan1 = pna.ActiveChannel ' set the active channel

chan1.SelectCalSet E_cal_ID, 1 ' set the calibration ID
chan1.ErrorCorrection = True ' Turn on calibration
Set fixture = chan1.Fixturing
GHz = 1000000000#

StartFrequency = Range("I70") ' get start frequency from the chart
StopFrequency = Range("I71") ' get stop frequency from the chart
NumberofPoints = Range("I72") ' get number of frequency points from the chart

Set measures = pna.Measurements ' create list of measurements
Set win3 = pna.ActiveNAWindow ' create window for plots
Set meas1 = pna.ActiveMeasurement  ' get object for active measurements

chan1.StartFrequency = StartFrequency * 1000000# ' set start frequency
chan1.StopFrequency = StopFrequency * 1000000000# ' set stop frequency
chan1.NumberofPoints = NumberofPoints         ' set number of frequency points
chan1.IFBandwidth = 1000 ' set IF bandwidth

Sync = False                    ' Grab one set of data and pause
chan1.Single Sync
set pna = Nothing ' close the network analyzer object

When the SAVE button button is pressed, the tool goes through a few quick checks before saving data.
The checkers include:
> data entry check - were the results cleared and a valid set of inputs entered?  (part number, production code, batch, and container)
> settings check - is the network analyzer set to the right start and stop frequencies
> continuity check - read S21 and S43 dB at the lowest frequency (typicall 10MHz) and check that the value is at least -3dB
> reflections check single-ended - single-ended reflections S11, S22, S33 and S44 are below -6dB max value over frequency, and they are consistent (within 1dB of each other at the lowest frequency)
> reflection check differential - check the quality of the device attachment to the fixture by checking SDD11 and SDD22 max value (-10dB or better) in the relevant frequency range

EXAMPLE CODE TO CHECK AND SAVE THE DATA:
******* Excel Macro **********
''' DATA CHECKERS
Set pnachk = CreateObject("AgilentPnA835x.application")
Set chanchk = pnachk.ActiveChannel
Set measureschk = pnachk.Measurements
Set measchk = measureschk(5)
s21 = measchk.GetData(2, 1)
Set measchk = measureschk(6)
s43 = measchk.GetData(2, 1)

startfreq = chanchk.StartFrequency
stopfreq = chanchk.StopFrequency

' CHECK CONTINUITY, THAT POINT A IS CONNECTED TO POINT B
If s21(1) < -3 Or s43(1) < -3 Then
    MsgBox (">>> Please check the PORT ORDER and the wire connections. <<< " & vbNewLine & vbNewLine & "(Through paths S21, S43 Fail continuity check.)" & vbNewLine & vbNewLine & "DATA WILL NOT BE SAVED.")
    Set pnachk = Nothing  ' close pna object
    Set chan1chk = Nothing
    Set measureschk = Nothing
    Set measchk = Nothing
    End
End If

' Reflections checkers, if the reflections vary too much across the ports, then the measurement is not accurate
Set measchk = measureschk(1)
s11 = measchk.GetData(2, 1)
Set measchk = measureschk(2)
s22 = measchk.GetData(2, 1)
Set measchk = measureschk(3)
s33 = measchk.GetData(2, 1)
Set measchk = measureschk(4)
s44 = measchk.GetData(2, 1)

mx1 = Application.Max(s11(1), s22(1), s33(1), s44(1))
mn1 = Application.Min(s11(1), s22(1), s33(1), s44(1))

If (mx1 - mn1) > 2 Then
    If MsgBox(">>> Please check the ground connection and signal path. <<<" & vbNewLine & vbNewLine & "(Reflections S11,22,33,44 show too much variation.)" & vbNewLine & vbNewLine & "DO YOU WANT TO FORCE A MEASUREMENT ANYWAY?", vbYesNo, "Confirm") = vbNo Then
        MsgBox ("Data file not saved.")
        Set pnachk = Nothing  ' close pna object
        Set chan1chk = Nothing
        Set measureschk = Nothing
        Set measchk = Nothing
        End ' if checker fails, stop the measurement
    End If
End If

' if the checkers pass, make a measurement sweep and save the data
filepath_s4p = Range("D7") ' get the directory path from chart
filename_s4p = Range("D9") ' get the filename from the chart

fullname_s4p = filepath_s4p & pair_filename
checklen = Len(Dir(fullname_s4p))  ' check if the file already exists
    
If checklen Then
  If MsgBox("S-parameter s4p file already exists. (" & fullname_s4p & "         Do you want to save a new file?", vbYesNo, "Confirm") = vbNo Then
    MsgBox ("S4P file not saved.")
    Range("cz1") = "0"
    End
  End If
End If

Range("D11") = "Measuring... Please Wait"   ' message to user

chanchk.IFBandwidth = 1000
chanchk.Single True ' do a single sweep and stop
Set meas = measureschk(1)
Ports = Array(1, 2, 3, 4) 'List the port numbers desired
meas.WriteSnpFileWithSpecifiedPorts Ports, fullname_s4p 'save s4p file


Set pnachk = Nothing ' close the network analyzer object

If any checker fails, the tool stops and does not save the measurement.
After the checkers pass, the tools saves the .s4p s-parameter file using a specific, standardized naming convention.
When the REPORT button is pressed, the tool grabs the most recent set of .s4p files, and creates a report showing unique tracking info for the device measured, PASS/FAIL results, and relevant plots.
By default, the report requires no edits before publishing. The user may add any custom edits as needed, especially for engineering experiments.

REPORT OUTPUT

BEFORE & AFTER TIMING

Time cost:
BEFORE
Setup measurement: 30 seconds x 8 = 4 minutes
Measure data and save files: 2 minutes x 8 = 16 mins
Extract CSV files and screenshots:         4 minutes x 8 = 32 mins
Copy and paste CSV files and screenshots into report = 15 mins
---------------------------------------------------------------------------
TYPICAL TOTAL TIME = 67 mins

AFTER
Setup measurement: 30 seconds x 8 = 4 minutes
Measure data and save files: 20 seconds x 8 = 160 seconds
Generate report: = 1 minute
---------------------------------------------------------------------------
TYPICAL TOTAL TIME = 8 minutes

After implementing this tool and workflow, lab throughput went from about 400 s4p's per week to about 1000 s4p's.  Another benefit was that the lab tasks became easier, so there was more time and energy for development work in addition to the production work.

It takes a noticeable amount of time to build the first tool, at least a few hours.  It may take longer to build the tool, than to do the task the old way a few times.  However, it pays back time by an order of magnitude each time the tool is used.  It only takes a few uses to pay back the development time, then it starts paying time dividends with every use !

PRODUCTIVITY CHART, BEFORE AND AFTER IMPLEMENTATION OF TOOL

What tools do you wish you had to do your job?
What if you wrote your own tool?