Hi Girish, That error represents that the Connector was unable to locate the target server associated with the URI/DNS record (url). Based on experience if this is reproducible it probably means that the target server is behind a firewall and we cannot access it.
Do you have any additional information? Is this still an active problem? Can you still* hit the sftp server from your local machine and using SFTP software? This last one is usually a surefire way to determine if some firewall exists and whitelisting needs to happen on the part of the sftp server.
... View more
Hi Tanayj. The product catalog name and description fields you mentioned are not passed into the hook for performance reasons. The majority of our customers do not utilize these data points so they would increase overall payload size mostly without benefit.
We do pass the Subscription Name and it should be available for you as part of the Liquid Configuration. In the template configuration section, if you click show Liquid Examples, it'll be underneath the Document Item category.
For your other fields you can make custom fields on the related objects, populate them with the information you want to access, configure them as part of the data the connector accesses underneath the Setup External Tax Engines page under Billing Settings, and then update your request template in the Tax Connector to reference these.
... View more
Add the field "FieldsToNull" as part of your Update task. From that field you should be able to select the field you are trying to blank out from a drop down, see screenshot. In the below example I've added FieldsToNull as a field to update against my Subscription object and then selected ContractId__c to blank out.
... View more
Preface: Monolith to Micro-Service Evolution
In the last couple of years, there’s been a design paradigm shift from monolithic architectures to micro-service models for web applications. As a monolithic application’s complexity increases, release cycles commonly increase in duration and decrease in efficiency; regression testing, compilation and deployment time are all impacted, which in turn reduces engineering velocity for releases.
Young startups are quick and nimble because they don’t have to deal with thousands - sometimes millions - of lines of code, so a Monolithic architecture approach makes sense for a new product launch. There’s a large amount of risk in spending months over-engineering an application stack when the product itself may never gain traction. But what happens when your product does gain traction?
A lot of people, including ourselves, are talking about the benefits of moving to micro-services, but few are talking about best practices for development velocity and maintenance of micro-services. With the advent of micro-services, there is significantly more overhead all around, physically on the infrastructure side and performance-wise due to latency and a desire to eliminate point to point communication between servers.
What kind of security, communication and data access frameworks need to be in place to ensure your engineering velocity speeds up instead of down as the number of micro-services your organization supports increases?
How do you best enable your team to quickly deliver a micro-service to production without everyone being a full stack engineer with production access?
We don’t claim to have all the answers, but we will share our story going through this evolutionary process and the decisions we’ve had to make thus far. We are hoping some of our lessons learned can be takeaways for other engineering teams trying to solve similar problems.
The Connect Marketplace Story
The Connect Marketplace started about two to three years ago as a monolithic Ruby on Rails application. The Marketplace is a platform that allows “applications” to be developed and integrated against your Zuora environments.
Figure 1 - Connect Markplace Apps
Functionally, we are trying to achieve something similar to all of the popular appstores out there: Apple iTunes, Google Play, Salesforce AppExchange are a few that come to mind. By allowing external developers to extend our core Zuora offering, the value Zuora provides can scale exponentially independent of the rate at which Zuora proper delivers platform and product features.
Connect - The Monolith
The Marketplace minimum viable product (MVP) was centered around our first “app”. Namely, a set of views, models and controllers that were baked into the rest of the Connect codebase that was designed primarily to control user authentication, application listings, provisioning of “installed” applications in Zuora environments, and infrastructure management at the admin level.
We received positive feedback almost immediately and were soon on the horn for releasing two to three additional “applications” using the monolith approach.
Our First Micro-service
When planning our fourth application, our technical requirements were different than in the past. Whereas past applications focused more on analytics and bulk processing of data, the new app required high responsiveness, high API throughput and would need to support spiky transactional behavior. Based on these requirements, we needed an isolated, standalone application stack. We tapped our team’s residential AWS guru, Matthew Ingle, to launch a new stack; ELB, a set of EC2 servers for the frontend and the backend and an RDS DB. Matt also wrote a quick deployment script that would pull code from the master branch of the new applications’ git repos and perform rolling restarts…. Success!
Our engineering team has since grown and Matt ended up owning, launching, configuring and maintaining our separate application stacks:
Each time a deployment was required, someone would swivel their chair over and tap him on the shoulder.
Each time there was some kind of exception thrown in the production servers Matt would SSH in and pull down the application logs.
Server crashed and didn’t self-heal? Matt had to manually launch a new server.
Server crash in the middle of the night? Someone named Matt was getting a phone call.
Need a new server stack? Get in line, put in a ticket, it’ll be available sometime next week.
Does the above sound familiar to any of you?
Very quickly, Matt was consuming most of his cycles supporting the rest of the team and became a scalability bottleneck. We knew it was time for a programmatic solution.
The Next Five Micro-services...
We need an easy way for our teams and users to provision their own infrastructure and preferably in a way that abstracts the need for technical knowledge of the underlying components (Servers, DBs, Redis, etc), to deploy their own code, to access application logs, to be able to monitor and benchmark their application performance. The key requirement for all of this is to enable folks without relying on key members on the team, and do it all without direct AWS access or production SSH access.
In the next part of the Zuora Connect Journey series, we’ll share details surrounding the software stack we ended up choosing to solve this scaling issue, and how we ultimately bridged the gap between our components.
... View more
With regards to the api loader, it validates the column headers in your file against the environment you are trying to load the file into. If any of the fields in your header do not exist in your target environment it will exclude them from processing as you can see in the log file. This is meant to assist system users that may use the same template for loading fields against multiple environments, specifically for custom fields that may be different across environments.
I can understand how this might cause confusion when mispelled fields are present but the process returns a "Success" message. We will investigate if this type of failure should terminate the process.
In order to ensure that you have the correct spelling for standard Zuora fields, we recommend that you do an export for the fields you are wishing to update and overwrite the values in this file before uploading.
... View more