Try rebooting any security appliances with the issue and that are set to passthrough or VPN concentrator mode. This resolved the issue for us.
Everyone on Meraki should be complaining to their account team and pushing for an RCA. Established tunnels aren’t supposed to be affected by cloud issues and it’s taken way too long to fix it.
Looks like bad code was submitted. I manage resolve it as we had used Passthrough by reloading it. There was no banner and when they decided to push one out it was green….red is better!
Has anybody noticed that after this update they pushed, devices that were once set to passthrough are now in routed mode?
I restarted my Merakis, but the tunnel still wouldn’t go up. Had to restart the other side to get it live. It wasn’t Meraki.
How do I know my MXs have this fix applied? Had a 30 min outage org wide because of this yesterday. Have rebooted my concentrators since.
Has anyone seen a RCA/RCO for this yet?
We now have to look into a backup to the dual-zone, failover MX config we have on AWS… thankfully the AWS setup we have is currently only in dev at the moment.
Ain’t no good if the vMX “HA” doesn’t HA.
I dont think there is any firmware notes but check https://status.meraki.net/ - The issue is resolved, assuming they pushed the updates through the cloud to all the MX/vMX’s
Looking for this also, if anyone drops a link when it comes out that would be super helpful
Was thinking what we do if this happens again, but you’re SOL, if all of the VPN brokering in the Meraki cloud is down no amount of HA or redundancy is going to help, all of your S2S is going to be down regardless of on-prem or any cloud platform. This is arguably the worse possible outage Meraki could have.
What bothers me is that it looked like the vMX locked up.
WTF kind of chicanery does Meraki have baked into this stuff that errors on their end lock up many of their customer’s products?
I opened a ticket with them requesting it. I’ll let you know if I hear back
We’re thinking of putting in an ASA and just using an IPSEC tunnel and route to that as failover.
All of our production workloads are in AWS and are full Meraki on-prem, we don’t have any non-Meraki VPN tunnels but I’m assuming those were down as well. We’re stuck if this happens again.
We actually had some meraki → ipsec tunnels go down too. Maybe coincidence…
Sorry I forgot to follow up. I received the report last week. Open a ticket with Meraki support and they can provide you with it.
What was in the report? Anything useful? Is it worth to ask them?
Personally I think it was.