Quantum Benchmarking and Noise Modelling
Lets digest pieces of chunks one by one- Benchmarking and Noise Modelling. And now put Quantum before the Benchmarking. So very fundamental question we should what is the benchmarking and why we need this? Here is the disclaimer for this long post- I explored this whole thing by myself and recently a preprint of related work is released [1]. Motivation behind writing this blog post is to explain our contributions.
Why we need benchmarking? Imagine, you are going to buy a processor- AMD Ryzen series or Intel X series. They are from different vendors but when you can buy only one. How will you compare which one might meet your requirements? Its pretty difficult task from a computing perspective. To do this, scientists run a set of programs and analyze the performance on different hardware. Based on the performance results now you can make decide that on this specific parameter this CPU is better or not. Researchers consider seven vital characteristics for a benchmarking suite [2]. They are-
1. If it can measure some vital features, like clock speed, latency- Relevance.
2. It should be applicable for a wide range of industry and academic usage- Representativeness
3. It should be comparable.-Equity
4. Anyone should be able to verify the performance based on benchmarking- Repeatability
5. Economical
6. Benchmarking test should work for scalable systems- Scalability
7. Lucidity
Typically industry and academia try to meet this characteristics. And in case of classical the main focus is put on CPU design. Now question is what would be situation for Quantum hardware?
It is quite challenging for several reason. To me, main challenge is still it is on embryonic stage, as a result, a lot of experiments are going on. For example, IBM quantum hardware is microwave superconducting based, while IonQ is laser based trapped ion technology, some other companies are focused on photonics. This diversity makes it difficult to test them against a set of programs. Moreover, algorithm performance varies depending on hardware. If we look the following image, we can say that different hardware using different types of benchmarking and each one has its own pros and cons [3][4]. But to evaluate performance of a system we benchmark the system.
When, I and Protik started studying benchmarking, we wanted to address current state of the art. Then we wanted to decide, what if we look at quantum hardware and set of programs with noise perspective. Then we studied what the noise make computation difficult for quantum. From there we chose 3 noises. Thermal relaxing, due to this qubits relax from higher state to ground state and make our computation difficult. Bit error, qubit always tend to flip from ground to higher state and vice versa. Depolarizing, for this it’s difficult for a qubit to trail a correct trajectory as a phase. We studied parameters of the noises and developed a noise model each noise. Then we selected a set of program on which we can test our noise model. To compare our noise model with a quantum hardware we choose IBM superconducting based qubits. Our main finding is that, the noise model we developed cant mimic the noise produced by IBM superconductor based hardware technology. The gap provides us hints that to achieve quantum advantages, our algorithms needs to be improved a lot, so that we can implement quantum algorithms for real-life applications. As real world is full of noise, if quantum states produced by a quantum algorithm cant survive higher performance efforts will be always daunting.
Acknowledge: Myself and Protik, express our gratitude to Dr Abdullah Ash Saki and Dr Omar Shehab for their mentoring and feedback.
[1] Sazzad, Sakibul Islam; Nag, Protik (2022): Quantum Noise And Measuring Quantum Distance For NISQ Circuits. TechRxiv. Preprint. https://doi.org/10.36227/techrxiv.21791987.v1
[2] Dai, Wei, and Daniel Berleant. “Benchmarking contemporary deep learning hardware and frameworks: A survey of qualitative metrics.” 2019 IEEE First International Conference on Cognitive Machine Intelligence (CogMI). IEEE, 2019.
[3] https://www.bcg.com/publications/2022/value-of-quantum-computing-benchmarks
