Integration

Speed Up Your Backend with Xano Transform

Summary

In the world of application development, efficient data manipulation is crucial for delivering smooth and responsive user experiences. At Xano, we understand this importance, which is why we've introduced XanoTransform, a powerful engine that harnesses the expression data type to streamline data manipulation processes. In this article, we'll explore the performance benefits of XanoTransform by comparing it with native function stack functions and JavaScript lambda functions.

Combining First and Last Names

Let's start with a simple example: combining first and last names into a single field. While straightforward, this operation can become time-consuming when dealing with large datasets.

Using a For Each Loop

// Query to fetch 25,000 records const results = await xano.db.query('customers') .limit(25000) .get(); // Concatenate first and last names using a for each loop const updatedRecords = results.map(record => { const fullName = `${record.firstName} ${record.lastName}`; return { ...record, fullName }; });

This approach took approximately 6 seconds to process 25,000 records.

Using a JavaScript Lambda Function

javascript // Query to fetch 25,000 records const results = await xano.db.query('customers') .limit(25000) .get(); // Concatenate first and last names using a lambda function const updatedRecords = results.map(record => ({ ...record, fullName: `${record.firstName} ${record.lastName}` }));

While faster than the for each loop, this method still took around 5 seconds to complete.

Using XanoTransform

// Query to fetch 25,000 records const results = await xano.db.query('customers') .limit(25000) .get(); // Concatenate first and last names using XanoTransform const updatedRecords = xano.data.set(results, '$.fullName', '`${$.firstName} ${$.lastName}`');

With XanoTransform, the same operation took just under 2 seconds, demonstrating a significant performance boost.

Standardizing Phone Number Formats

Next, let's explore a more complex scenario: standardizing phone numbers and removing extensions from user-generated data.

Using a For Each Loop

javascript // Query to fetch 25,000 records const results = await xano.db.query('customers') .limit(25000) .get(); // Standardize phone numbers using a for each loop const updatedRecords = results.map(record => { const fixedPhoneNumber = record.phoneNumber.replace(/\D/g, ''); const formattedPhoneNumber = `(${fixedPhoneNumber.slice(0, 3)}) ${fixedPhoneNumber.slice(3, 6)}-${fixedPhoneNumber.slice(6)}`; const { phoneNumber, phoneNumberExt, ...rest } = record; return { ...rest, fixedPhoneNumber: formattedPhoneNumber }; });

This approach took approximately 20 seconds to process 25,000 records.

Using a JavaScript Lambda Function

javascript // Query to fetch 25,000 records const results = await xano.db.query('customers') .limit(25000) .get(); // Standardize phone numbers using a lambda function const updatedRecords = results.map(record => { const fixedPhoneNumber = record.phoneNumber.replace(/\D/g, ''); const formattedPhoneNumber = `(${fixedPhoneNumber.slice(0, 3)}) ${fixedPhoneNumber.slice(3, 6)}-${fixedPhoneNumber.slice(6)}`; const { phoneNumber, phoneNumberExt, ...rest } = record; return { ...rest, fixedPhoneNumber: formattedPhoneNumber }; });

The lambda function approach took around 4 seconds, a significant improvement over the for each loop.

Using XanoTransform

// Query to fetch 25,000 records const results = await xano.db.query('customers') .limit(25000) .get(); // Standardize phone numbers using XanoTransform const updatedRecords = xano.data.set(results, '$.fixedPhoneNumber', '`(${$.phoneNumber.replace(/\D/g, '')})`') .set('$.fixedPhoneNumber', '`(${$.fixedPhoneNumber.slice(0, 3)}) ${$.fixedPhoneNumber.slice(3, 6)}-${$.fixedPhoneNumber.slice(6)}`') .unset('$.phoneNumber') .unset('$.phoneNumberExt');

XanoTransform completed the same task in just over 2.5 seconds, outperforming both the for each loop and the lambda function approaches.

Grouping Users by Company

In our final example, we'll group users by their associated companies, creating a nested data structure.

Using a For Each Loop

javascript // Query to fetch 10,000 records const results = await xano.db.query('customers') .limit(10000) .get(); // Group users by company using a for each loop let fixedData = []; const uniqueCompanies = [...new Set(results.map(record => record.company))]; for (const company of uniqueCompanies) { const users = results.filter(record => record.company === company); fixedData.push({ company, users }); }

This approach took a staggering 16.5 minutes to process 10,000 records.

Using a JavaScript Lambda Function

javascript // Query to fetch 10,000 records const results = await xano.db.query('customers') .limit(10000) .get(); // Group users by company using a lambda function const fixedData = [...new Set(results.map(record => record.company))] .map(company => ({ company, users: results.filter(record => record.company === company) }));

The lambda function method completed the task in just under 2.5 seconds, a massive improvement over the for each loop.

Using XanoTransform

// Query to fetch 10,000 records const results = await xano.db.query('customers') .limit(10000) .get(); // Group users by company using XanoTransform const fixedData = xano.data.indexBy(results, '$.company') .unique() .map('$.key', '$.value.map(user => ({ company: $.key, user }))');

XanoTransform outperformed both approaches, completing the operation in just over 1 second.

Using Aggregate Returns (Bonus Example)

// Query to fetch data using an aggregate return const results = await xano.db.query('customers') .groupBy('company') .addOn('users', { return: 'ids' }) .get(); // Fetch user data using the addOn const fixedData = await Promise.all(results.map(async result => ({ company: result.company, users: await xano.db.query('customers') .where('id', 'in', result.users) .get() })));

While not the focus of this article, it's worth mentioning that Xano's aggregate returns can also provide excellent performance when working with data stored in your database. In this example, the operation completed in just over 2 seconds.

Conclusion

As demonstrated through these examples, XanoTransform offers a significant performance boost when manipulating data, especially with large datasets. While native function stack functions and JavaScript lambda functions are viable options, XanoTransform's expression data type provides a more efficient and streamlined approach.

Moreover, XanoTransform truly shines when working with data sources beyond your database, such as CSVs uploaded by users or external APIs. Its ability to transform and manipulate data from various sources makes it an invaluable tool in your development arsenal.

We encourage you to explore XanoTransform and harness its power to unlock lightning-fast data manipulation in your applications. If you have any questions or need further assistance, feel free to reach out to us through the Xano community, support chat, or by leaving a comment below.

Happy coding!

This transcript was AI generated to allow users to quickly answer technical questions about Xano.

Was this helpful?

I found it helpful

I need more support