MEANIE – The Conception

The blog post is an announcement to begin a project called MEANIE – which stands for MongoDB, Express, Angular, NodeJS Integrated ECommecre.  The Concept is to create an Open-source platform for E-Commerce Based on these technologies.

Why?

Although my primary career revolves around Salesforce development and consulting, I always have an interest in Web Development in all aspects around me.  For the past year, I have been fascinated with NodeJS and the frameworks around it since I have developed and deployed a few solutions that integrate to Salesforce that is hosted on Heroku.  I have experience around e-commerce, as many profitable side projects over the years allowed me exposure to various e-commerce platforms. I figured this type of project would not only allow me to fully learn and explore NodeJS but would help me to bring some E-Commerce project I have on the side up-to-date.

Aren’t Their Other Frameworks / Platforms?

Sure, I have tried to find them.  So you have Reaction Commerce. It is based on ReactJS.  I don’t want to learn ReactJS. Nothing against that framework, I just don’t want to learn it. Then there is Schema.io. For me, the problem with Schema.io is that your data lives somewhere else. What if that service is down? It seems like a great scaffolding, but I just like to have full control over my whole system – data, middleware, and front-end.  It is an intriguing system, so I’m thinking I will create a pluggable interface to MEANIE to support this. I thought Traider.io was the answer, but sadly it seems that project has gotten nowhere.  I forked the repository but found quite a few problems with deprecated npm packages and the fact that they did not fully follow the Twelve Factor App Principles, which made it really hard to fix.

Bottom line, there seems to be a lot of projects around, but they are either hobbies, paid, or only support a specific stack with ‘new’ or unknown components. Hopefully, my project doesn’t fall by the wayside, and I am hoping by keeping it open for collaboration, it won’t.

Well enough laying out the vision – time to get started!

 

Advertisements

Process Builder Bulkified? Not so much…

Summary

You have created a Process Flow using Process Builder.  The Process works fine when processing a record one at a time.  But whenever changes are made to a number of records (such as Dataloader or a mass update from another process), your Process fails with “Too many SOQL Queries”.

Introduction

I love the way Salesforce constantly releases new tools to make everyone’s job easier and more streamlined.  The introduction of the Process Builder seemed like ‘manna from heaven’ with the promise of relieving developers from creating tedious triggers that did mundane things simply because workflow could not handle child relationship updates or other seemingly simple tasks. It gave hope to the hopeless administrators who pleaded with developers to finish their ‘one simple trigger’, or complete their silly ‘Visual Flow Plugin’.

But the release, and consequently the update provided in Winter ’16 did not live up to the full expectations of the people.  The initial release handled logic on ‘one record at a time’, which seemed great -if everything in your Org happened one….record….at….a….time. “When does that ever happen!” – many administrators screamed from the dungeons of their cubicles, as developers snorted “great, useless” in their murmurings rummaging through Salesforce help articles and seeing this limitation. The following release promised to “Reduce Chances of Hitting SOQL Limits in Processes“, but instead of ‘everything just working’ seamlessly, now the ‘Too many SOQL Queries’ seem to happen randomly.  What gives?!

Details

So – in the Winter ’16 release, and the corresponding solution to the idea that was posted around ‘bulkifying’ the process, Salesforce has the following verbiage:

The PM pointed out there is often a misunderstanding of what is causing most use cases to hit limits. The operation may be to create a bunch of records, but the queries associated are actually the main cause of limit errors. In such cases, you are hitting platform limits. At this point, Process Builder & Flow are as optimized and bulkified as possible. For those purely reacting to the details captured in the release notes, we will be updating that content to make it clearer.

Clear as mud, right?  Here is what they are trying to state:

SOOOOOOO

For inserts and updates defined within the ‘Actions’, we will bundle as many of these records together as possible (based on the Criteria you defined) to perform these actions… This will reduce the number of DML Statements that we execute within a transaction on your behalf… more on that later…

For the magic ‘decision diamond’…. (which this is where your SOQL Query Limits happen) they are trying to state the following:

AND…….

In order to determine what records can flow through the process you created, we will need to query these records.  We use the Standard Apex Language on the backend, so the queries we use are restricted to those limitations. As we build the queries, we will try to build them is such a way to limit the number of queries we make, but in the end – most likely, each criteria will result in 1 SOQL query per record….

So this means, if you have 4 Criteria in that diamond, more likely than not, for each record, 4 SOQL statements will need to be issued to determine if that record can follow through to the ‘Actions’ to be executed for that record. Now, as it is evaluating each of the records, it will store them in kind of a buffer, or queue, and once all of the records are evaluated, it can determine which records (in bulk), so send to the ‘Actions’ for the true evaluation of that diamond, and which records to send to the ‘Next Criteria Evaluation’ (if one exists).

Continuing on the example from above, this means that if you are dataloading records that are to be evaluated by the Process that has 4 Criteria listed, most likely, in order to avoid the SOQL Query limit (if it is – lets say 100 Queries within a single transaction) set your batch size to 25.

Additional Information

So does this ‘SOQL Query’ statement made above apply to each ‘Criteria Diamond’ in our process? Yes, yes it does. How can we get around it? This is where the ‘Headless/Autolaunched Visual Flow’ comes in – the Visual Flow that you can invoke from Process Builder processes. Salesforce has made advances in this area to allow the headless/autolaunched flows to be executed multiple times under the right circumstances to avoid the SOQL limits exception.  See the Salesforce help menu on setting this up. There are also other articles that tackle this as well.  Be sure to search the Stackexchange Salesforce Category as well for more complex examples, problems and solutions.

Summary

So in closing, Process Builder is an important evolution to the distribution of architecting and implementing business logic from the developer to the administrator on the Salesforce platform.  Sure, its not perfect yet, but hopefully the information above can help stop your headaches related to finding seemingly ‘intermittent’ SOQL Query errors in your Salesforce processes.