Sunday, December 9, 2018

What hash does Go modules (vgo, go mod) use for verification?

As mentioned in a previous blog, git's hashing mechanism isn't great.  At the time of this writing, the replacement is still not fully ready, let alone used. In my humble opinion, securely verifying code should be a primary function of git.  Git historically has been indifferent on this point.  It's argued git shouldn't be as it isn't security focused, while frustratingly containing a built in function to do just that. This means secure infrastructure must be a layer separate and above git. Languages or other ecosystems that use git must build their own systems.  

Go had no standard means of addressing this issue until Go modules.  Thankfully, Go wisely knew not to delegate this vital security feature to git.  This also allows seamless employment of other versioning system like mercurial or bazaar.  

A project's `go.sum` file can reference an external project like this: v0.0.0-20171218180944-5ea4d0ddac55 h1:jbGlDKdzAZ92NzK65hUP98ri0/r50vVVvmZsFP/nIqo=
In this example, the project didn't have a manually created version number set as a git tag, so Go created one.  `5ea4d0ddac55` is the first part of the git commit SHA-1 hash sufficient in length to reasonably avoid collisions.  A hash is nice to have as a component of the version number, but Go does not use this for anything security related.  Why then use a hash for a version number?  Time isn't sufficient as version should be related to what the code is.  By including parts of the commit hash, small changes will result in a different version number.  

But the line doesn't stop with the version.  The line continues and references "h1".  What an improvement!  Go modules is already anticipating using different hashing methods in the future.  What is h1?  Go modules uses SHA-256 as a hashing algorithm for the verification of code and econdes the hash in Base64.  More specifically, Go modules compresses the whole directory as a `.zip` and then hashes the results.  The local cache directory for the results are kept in `$GOPATH/pkg/mod`.  The file in the cache ending in `.ziphash` contains the same hash as in the `go.sum` file in a given project.  

Here's a couple relevant links to how this is accomplished:

Thanks to Go's new module system, gophers no longer have to worry about what git is using. Go's verification mechanism is totally separate from external versioning tools and Go modules uses git only for versioning.

Thanks Go!

Monday, November 19, 2018

How to repair a Electrolux canister vacuum with a dead head

I was generously given a Electrolux Ultra Flex canister vacuum bringing the total owned by my extended family to three.  Electrolux has since released a new model after mine was purchased.  

It sucks well, it cleans very well, it's decently quiet, fantastic on pet hair, it's great on stairs, and the vacuum itself is easy to clean.  

Overall, it's probably the best vacuum I've ever used, except one has had a few issues.  The clip for holding the wand upright broke, the retractable cord doesn't stop recoiling forcing me to use a clip on the cord when vacuuming, and worst of all the the head stopped working after a few months.  No lights, no roller.  The first two issues aren't a deal breaker, but a dead roller is a show stopper. 

I troubleshooted to an electrical issue where the detachable hose meets the canister.  The electrical collar had a bad connection.  I attempted to clean it of any debris but that didn't seem to help. 

I called Electrolux and they sent out another hose assemble with no problems.  After a few weeks the new hose died as well.  Since two hoses had died and Electrolux just released a new model I figured there was a design problem with the older model I owned.  At this point I decided to fix it myself. 

A few warnings:
  1. This is an electrical modification and you should know what you are doing.  
  2. The canister collar will no longer be able to spin.  This is a hard wire fix.  If this isn't suitable for your vacuuming style, you will need to find another solution.  This isn't really a concern for me.  
  3. Make sure this is the issue.  It was easy for me because I could interchange parts with other vacuums.  Also by wiggling the hose or holding it down it was obvious there was an issue with the base of the hose.  
  • Wire cutter
  • Wire stripper
  • Phillips screw driver
  • Three 1/2 stainless steal screws
  • Small gauge electrical wire insulator, like shrink tube or electrical tape/paint
  • Drill with a small drill bit.  
Open the collar port.  There's a single black screw to do this.  Then gently slip the outer collar from the inner.  An o-ring might slip out.  Be sure not to damage it or loose it. 

Slipping off the outer collar reveals two copper rings.  Removing the copper rings reveal the wires connected to the hose.  Cut the wire off the copper ring (and be sure not to cut the wire off from the hose!). The wires under the copper ring are bare and you will need to be insulated.  Put the o-ring back and slip on the outer collar feeding out the two wires attached to the hose.  

Now you're ready to hard wire.  The collar port contains two wires that attach to the main canister body.  Wire the black to black and white to white.  Use an insulator to make sure there's no bare wire that could short circuit.  

Now that it's hard wired, the collar _must not_ spin.  I drilled three small holes from the outside and used 1/2 inch stainless steal screws.  This will stop the base of the hose from spinning so that the new hard wired solution won't be ripped out. 

Finally, slip the collar port back on and attach the black screw.  That's it!  If you've had this problem like I've had with two of mine, that should fix the issue. 

Happy vacuuming! 

Friday, September 21, 2018

Gravity Wells are Sombreros

You've probably seen an illustration of a gravity well, something like this:

Or maybe you've seen one with Earth like this image:

Whenever I saw these gravity well depictions in school I mistakenly assumed they depicted gravity.  They don't.  These gravity wells depict gravitational potential, not gravity.  Gravitational potential tells you how much time is warped and how slow local time will progress due to general relativity.  It is not, however, correlated to what local gravity is.  These "gravity wells" are better thought of as "gravitational potential wells".  

Another problem is that these illustrations work fine for distances far away from massive bodies, but in regions of massive space, it is not how gravity works. For example, Xkcd's depiction relates planet radius to gravity.  That depiction shows gravity up to the surface of the planets, but then sneakily substitutes gravity with planet radius.  The presentation obscures that no information about gravity inside of the planets is provided! These depictions we've all seen our whole lives all share one blunder: they do not show how gravity works in massive bodies.  

So then, what do gravity wells look like near and in massive bodies? To illustrate, let's envision an empty area of space with a one light year radius and a single celestial body, for example Earth, in the center.

To establish current understanding, in this thought experiment where is the least and  most gravity?

Based on the gravity well depiction, one might assume someone standing on the edge of this miniverse would experience the least gravity and someone standing in the middle would experience the most.  

That's a reasonable answer based on previous depictions, but it's not true.  

Where is the most gravity in Earth? Could it be the Dead Sea, the North Pole, the Mariana Trench, or Mt. Everest?

Consider this graph of Earth's gravity according to the Preliminary Reference Earth Model (PREM).  The area of most gravity is almost 3,000 km below Earth's surface about halfway to the core. The point of least gravity is the center of the planet. 

The points of both most and least gravity are both below the planet's surface!

The answer to our previous hypothetical was wrong. A miniverse with an Earth like mass has the point of least gravity at the center and the area of most gravity below the body's surface.  

To match traditional gravity well depictions Earth's gravity graph can be flipped so that low gravity is up and high gravity is down. Mirroring the graph then shows a path down to the core and back up again. The result is a sombrero cross section of Earth's gravity. Note the highest point in gravity well, the point of least gravity, is inside of the planet and not on the edges. The edges approach zero, but will never reach it.

How does this relate to general relativity and time dilation?  Although said frequently, gravitational time dilation is not dependent on acceleration.  
 Time dilation is always dependent on velocity.  This is obvious for special relativity.  In the case of gravity wells, time dilation is dependent on escape velocity; acceleration isn't needed in any of the calculations.  The higher the value for escape velocity, the larger the time dilation.  The point of the most time dilation is the center of Earth, while the least time dilation is the point furthest away from Earth.  

To match the traditional 3D depiction, Earth's gravity well looks something like this.

Gravity wells are not just dips.  Gravity wells are sombreros.

Wednesday, March 21, 2018

It's JOSE, not JWT (A Pedantic Complaint)

You might have heard about JWT's, but the JWT specification is just about claims as a payload inside of a JWS, which is apart of the JOSE specification. 

In a small series of RFC's, 7515 to 7519, JOSE (JSON Object Signing and Encryption) is defined as a standard that uses cryptographic functions for communication across applications using JSON.  JOSE is an effort to modernize a hodgepodge of standards and provides a basic framework for future cryptographic applications all while using the popular and familiar JSON standard.  JOSE isn't alway concrete with its recommendations and instead sometimes prefers basic guidance for applications seeking a starting point.  JOSE is a needed and welcomed addition to a long history of web standards published using the Internet Engineering Task Force RFC process.  
JOSE is broken into two specifications.  JWS (JSON Web Signature) is for signing and integrity protection.  JWE (JSON Web Encryption) is for encryption.  For some undivined reason JOSE's introduction resulted in much of the web referring to the whole standard as "JWT" (JSON Web Token), a small section of the larger standard.  Perhaps it's because of the novel way JOSE was popularized on the web with cookies.  JWT claims used in cookies can free servers from remembering session information by trusting the cryptographic signature, originally signed by the server, provided by the client.  This novel usage is a small part of a much larger standard, and even this usage highlights the power of JWS.  There's nothing special about JWT in a generalized case, as it is simply just a payload.  
Perhaps the name "JWT" is popular because of the way JOSE introduced itself in its RFC's. The first RFC in the series concerns JWS and doesn't really explain what JOSE is.  Instead the reader is told a JWS header is called a "JOSE header" and that everything else, its payload and signature, is called a "JWS payload" and a "JWS signature" as one would expect.  Why would only the header be named "JOSE"? The reader using these clues is initially left to infer JWS is a subset of JOSE.  In later RFC's JOSE clearly becomes the given name for the standard as a whole.  For example RFC 7520 is titled "Examples of Protecting Content Using JSON Object Signing and Encryption (JOSE)".  This is also made evident in drafts which include JOSE in the name as it is the name of the IETF working group.  This information is lost to readers new with the final publishing.  
Now back to JWT, a JWS or JWE encapsulates JWT.  The JWT specification notes that JWT's are just claims in a JWS.  
   JSON Web Token (JWT) is a compact, URL-safe means of representing
   claims to be transferred between two parties.  The claims in a JWT
   are encoded as a JSON object that is used as the payload of a JSON
   Web Signature (JWS) structure or as the plaintext of a JSON Web
   Encryption (JWE) structure
JWS is the real star of the show for most web applications, and all these pieces fit together under JOSE.  

Friday, March 16, 2018

Mormons, Polygamists, and Evangelicals: My Experience with the iWorks Scandal


I worked for Jeremy Johnson in Utah. He stole millions from people by charging customers for "free trials".

Mormons, Polygamists, and Evangelicals: My Experience with the iWorks Scandal

In late 2008 I moved to Ephraim, Utah in the Sanpete valley for a few months one year after graduating from high school.  I was 19 years old.
I lived in the second story of a coffee shop caddy-corner from Snow College in what was once an old polygamist house. To quell any doubt of its history, the owner loved showing guests a false wall in a bedroom revealing a secret space, furnished with a small wire bed, where a polygamist hide wives during US marshal inspection.
“It’s Mormon forbidden but good!” the owner would jest pitching a cup of coffee. The shop was run by a determined group of evangelical Christians aiming to provide diversity in the homogeneous Mormon environment to the young student population attending Snow. There were Mormon church buildings every few blocks in the Mormon grid-planned city and the coffee shop was one of the few non-Mormon religious groups in the region. I was personally ex-Mormon having determined Mormonism wasn't for me at a young age.  I had become acquainted with the coffee shop years previous by chance on a MySpace page.  I moved to Utah with the hope to learn more about Christian ministry as I had hoped to eventually to enter ministry.  My Mormon childhood had exposed me to enough religion to think ministry was my best chance at having a positive impact.  I planned on finding work and being apart of the coffee shop efforts.  This "internship" would be apart of my schooling.


Ephraim sits an hour south of Provo with large stretches of open valley and farms between. During my first trip to Provo a few large white buildings caught my attention.
“Why are there hotels out here in the middle of nowhere?”
“Those aren’t hotels,” a friend replied. “Those are polygamist homes.”
Once a friend whispered at the Walmart in Ephraim while pointing their nose inconspicuously at the checkers, “They’re all married to the same man”. I later confirmed with others that indeed this was true.

Gays, Google AdWords, and an Illegal Alien: The iWorks Whirlwind

As a broke 19 year old, a call center selling Internet products was the most appealing employer in the Sanpete valley.  After a short interview I was hired.  They noted my Coloradoan accent was perfect for the job.  My calls were related to Google educational products.  
My first shock came shortly after being assigned a desk. I was at the end of a row and only had one person next to me.  
“Next week is my last week”. The young woman who sat next to me said. “They found me.”
I raised my eyebrow with obvious confusion. “They found you?”
“I get deported next week. I have no family in Mexico, I don’t know Spanish, and I’ve lived here my whole life. All I’ve ever known is Sanpete. I don’t know what I’m going to do.”
I never would have guessed.  She was a perfect, respectful, hard working American.  She told me she was brought to America when she was only a few months old.  She was gone the next week and was perhaps 20 years old.  
Many of the managers were apart of a relatively liberal polygamist group in Manti with some managers even married to the same men. Many of the workers were a part of another polygamist group in the valley. There were also a few Mormons and a handful of non-Mormon Christians.  
The first week on the job my supervisor told me, “Management likes you Zach. I overheard one of the managers saying, ‘His voice is as smooth as butter.’” My metrics were outstanding.  I was a young kid who had loved everything computers and spent most of my free time playing with Linux.  I was able to manage their workflow intuitively and my metrics showed it.  
Later, in a private closed door review, that same manager said with a thick Sanpete accent directly to me what she had already told my supervisor, "Your voice is as smooth as butter.  If I wasn't married I'd snatch you right up."  She was at least a decade or two older than her sister wives and perhaps double my age.  "Was she just horny?" I wondered.
The overseas call center was on a Filipino island and we were told that not only were all male Filipino employees gay, but in fact, the whole island was gay.  Was this the biased perception of the polygamist management or a poor retelling of a poem by Sappho of Lesbos?  The truth stayed hidden as I never dared to ask our Filipino coworkers directly, but they were very friendly and overjoyed to talk with Americans.  
“Hi sir Zach! It’s so good to hear you, sir Zach!” I would hear ecstatically many times a day. We would laugh at their cheerful insistence on always calling us "sir".  


One of my roommates was Gabe, an ex-Mormon like me. We meet before during my previous summer visits to the Sanpete Valley and always enjoyed his cheerful company.

Gabe was in need of work, but had terrible seizures from neurosarcoidosis at night.  Sleep proved a difficult burden to overcome in holding a day job. He had been let go from various places around the Valley who saw Gabe as flaky and undependable.  As his roommate I knew the grimmer reality of his condition.
iWorks was in desperate need of help and I asked them to hire Gabe.  I explained his condition and they were willing to deal with the unexpected absences Gabe’s seizures caused. “Show up when you can and we’ll pay you for that."

Scandal Uncovered

One day a Filipino employee transferred a call to me. Normally there were no issues as we would issue refunds liberally, but this particular customer insisted on speaking with a supervisor despite a full refund. As English speaking Americans, most American positions were “supervisors” of the overseas staff.
“Here’s the link to the signup page”, I said to the customer.  
They responded, “No. this was not the link. I signed up on a different link.”
"This is the link I have sir."
“No, you are wrong. I read everything thoroughly. The page promised not to charge my credit card” They said stressing every syllable. “Please, find out what they are doing and stop them from hurting people.” They paused, searching for words.  “Get to the bottom of this and stop them from hurting more people.”
I was moved by the genuine plea for correctness so I started to search.
iWorks had given us a large list of links to all the company’s products. During training I had also seen a map of “target regions” which was the contiguous United States with the glaring exception of Utah itself. iWorks did not sell products to Utah, and appeared to exclude Utah from its AdWords campaigns. I had always thought this was strange, so I searched Google for terms related to our products over a VPN to outside the state. Google products, health, supplements, fitness, weight loss training, and “acai berry”. I quickly found an ad for one of our product pages.
He was right. The page iWorks showed to customers was totally different from the links given to employees.   The only way we would have discovered this was by using an Internet connection outside of the state.  
Nearly identical to the links given to employees the customer facing page explicitly promised to not charge after the trial period. The page the employees were given said exactly the opposite. The link the company provided in emails and to employees was purposely deceitful. Management had been purposely deceiving employees with false pages which we were then giving to complaining victims. Even more concerning, from my workstation I could not access the URL, but it worked fine over the VPN. They were purposely blocking links inside the building and perhaps from IP addresses in the state. Even if a customer had sent us a link, employees would never have a way to view it, and would assume the customers were wrong. No outside emails were allowed.  There was little chance an employee could ever see a screenshot and even then, how would they know it wasn't doctored?
I immediately reported this to my supervisor, wrote an email to management, and quietly spoke with the trusted friends I had in the office.
“I’ll issue refunds to everyone who calls in!” Gabe exclaimed when I told him my discovery and showed him the links.
“You'll probably lose your job, Gabe.”
“I don’t care, this is wrong. I’m giving everyone a refund that calls in. I won’t let them cheat old grandmas like this!”
I was happily stunned at Gabe’s fidelity. This was the only job Gabe could hold in the valley, and he didn’t give it a second thought. He refused to do anything wrong. He refused immoral compensation.  There were others in the call center that felt the same way.  
One of my other friends was not so gallant.  “I can’t lose this job. I have to provide for my wife. I’ll just do what I’m told and trust that it’s okay with God.”  I was disgusted with his appeal to God to bless  willing compliance. He was able bodied, mobile, and could find work elsewhere if it came to that.  It was a final blow to a strained friendship and was a stark contrast to Gabe’s intrepid commitment to justice.  This friend was one of the reasons I had been living in Utah hoping to learn more about ministry.  Without a strong sense of morality, I didn't desire to work with them on any future endeavor.

Cover up

I got a message from my supervisor the next day. “Management wants to see you.”
“It’s been fun.” I thought as I entered the manager's office.
“We’ve been impressed with your performance Zach. We are promoting you and giving you a raise, a bonus, and making you a supervisor trainer over one of our new programs.” My mouth dropped. I had only been there a few weeks and they were planning on promoting me?  Did they not notice the volume of refunds we were giving?  Our meeting was short, and as I was leaving, she remarked, “Oh and one more thing, about your email, we’ve fixed the problem, thank you for bringing it to our attention. It was a mistake from one of our old, out of commission sites. We’ve taken it down, and we'll be sure that we'll make it right with the few customers affected who signed on from that page.”  I checked that night at home, the secret pages I had found were removed. Just maybe it was a genuine mistake.  My new duties removed me from working with customers or product links directly and I wondered if this was the reason for my promotion.   Regardless, if they tried it again, I would stay vigilant.  
In a couple weeks iWorks release a new acai berry product. Again, I searched Google for terms related to our products over a VPN.  Again, I found landing pages purposely omitting charge information and promising customers that they would not be charged.  The copy distributed internally that had been given was the exact inverse with explicit information about credit card charges. Again, employees could not access customer URLs from inside the call center.  This was definitive. These pages were new for the new products. This was the final nail in the coffin. They knew exactly what they were doing. They were purposely lying to their customers and charging millions.  
I told everyone that would listen to me in the call center and showed them printouts of my proof.  I wrote another email to my previous supervisor and CC’d management and I turned in my resignation.  I felt like I had little reason to stay so I left Utah in the beginning of 2009, moving back to Pueblo, Colorado, my home.


I regret not whistle blowing, but as a 19 year old I didn’t understand the full depth of iWorks deceit. I didn't know whistle blowing was a thing. I didn't know what they were doing was illegal as they had the blessings of the banks and credit card companies.  I only knew what they were doing was morally wrong and I wouldn't be apart of it, just like Comcast's attacks on net neutrality.   Thankfully, Jeremy Johnson, the owner of iWorks, was rightfully sentence in a federal case in what is being called the “biggest political scandal in Utah history”.  I hope he's removed from doing any harm for a long time.  
My friend Gabe, 31, of Ephraim, Utah, passed away unexpectedly on October 15, 2014 from complications with neurosarcoidosis.  I will never forget Gabe's commitment to doing the right thing in face of the vacuous unknown.  

Friday, February 9, 2018

Do not use git's built in gpg signing until git uses something other than SHA-1

If you trust SHA-1 and think it's secure then this post isn't for you.

If you think SHA-1 has been weak for years, is not sufficient now, and is dangerous moving into the future, this post is for you.

Git's online documentation describes how to sign a git commit.
$ git commit -a -S -m 'signed commit'
The massive elephant in the tiny git security room is that SHA-1 is no longer considered secure, and has been considered weak for a long time (about 13 years).  Bruce Schneier warned in February 2005 that SHA-1 needed to be replaced.  Git development didn't start until April 2005.  Before git had started development, SHA-1 was identified as needing to be deprecated.

When signing a git commit using the built in gpg function the project is not rehashed with a secure hash function, like SHA-256 or SHA3-256.  Instead, gpg signs the commit hash directly.

And yes, this commit hash is still a SHA-1 hash.   It's not signing the result of a secure hash algorithm.

A signing algorithm is only as good as its weakest point.  Just because something is signed doesn't magically make it secure if the underlying hash algorithm is insecure.  Worse, if a secure hash algorithm, like SHA-3, hashes the result of an insecure hash algorithm, like the git commit hash, the result is still insecure.

To make signing git code secure, the SHA-1 hash should be discarded and everything must be rehashed with a secure algorithm before signing.  This however is not yet standardized in the git world.  Anything one does securely will not be relevant for existing git infrastructure.  Go ahead and sign your code using something secure, your square peg won't fit into git's infrastructure.  Anything secure will need infrastructure over and above git to ensure security.

Git is moving to support secure hashes, but it's not adopted yet.  Even when it is, it will take time for the infrastructure to catch up.

So what's so bad about signing a SHA-1 hash?  If an attacker finds a collision they can have two copies of a repository, a good version and a bad version both with the signature's blessing. Google's security blog has better examples, but for simplicity:

              SHA-1 ( good code )     = hash123
              SHA-1( hacker code )   = hash123

If the hash "hash123" is signed with with git's built in gpg functionality, not only is the signature valid for "good" code, but it could be valid for "hacker" code.  Developers on the other side of the wire must have systems above and beyond git that prevent such attacks.

A malicious agent can commit a "good" version of a repo, wait until the commit is added to security sensitive repos, and then push the "bad" version.  Bingo.

With specific commit hashes you cannot verify the the version you are building in production is the exact same code as the one a security team audited prior to release without trusting systems external of git.

Alternatives?  Don't use git's built in gpg signing functionality.  Instead, compress your repo and sign the resulting file manually.  Anyone on the other side of the wire will need to familiarize themselves with this process, manually verify, or use a tool outside of git to verify.