Setting Standards: Escorting Code Generation in the Age of Big Models
AI Security Working Group "Royal Security" series of activities and generative AI code security standard preparation seminar was successfully held.
Representatives from the Institute of Information Engineering of the Chinese Academy of Sciences (CAS), Chianxin, SZXS, Tencent Jubilee Labs, Tianrongxin, Smart Spectrum AI, Nanjing Institute of Product Quality Supervision and Inspection, Xiaopeng Automobile, Baidu, Ant Group, Peking University, Kodak Xunfei, the Institute of Information and Engineering of the Chinese Academy of Sciences (IIE of CAS), and State Science and Technology Corporation (SSTC), participated in the seminar.
This workshop summarized the stage work and exchanged the next step focus. The meeting conducted in-depth discussions on generative AI code security force standard compilation and other related topics, etc., and shared the views of the industry and enterprises. The meeting pointed out that it is necessary to further implement the work of the AI safety working group, continue to make efforts in technical research, testing and evaluation, ecological construction, etc., to promote the development of a series of standards for the safety of AI technology, and generate a safe and reliable AI industrial ecology.
An Hui, Deputy Chief Engineer of China Electronic Information Industry Development Research Institute, said in his speech that with the rapid development of AI technology, generative AI is increasingly widely used in various fields, bringing us unprecedented convenience and efficiency. However, at the same time, the code safety issue is also becoming more and more prominent, which has become a key factor restricting its healthy development. Therefore, it is of great significance to strengthen the construction of generative AI code security and formulate perfect security standards.
Zhang Yong, vice president of Chianxin Technology Group Co., Ltd. said in his speech that big model technology is a new technology leading the fourth industrial revolution, and its application and development will certainly be the key work in the next few years. At the same time, with the development of related technology, we also need to solve the security problem of big model. On the one hand, there are various laws and regulations being discussed and launched by the government; on the other hand, it is more important that the industry itself carries out the construction of standards and norms.
Xin Pengjun, editor-in-chief of Communications Industry News, said that in China's vast, dynamic and distinctive characteristics of China's digital economy, AI security is crucial, and the development of relevant AI security standards will promote China's industry in the era of AI big sea voyage to go far.
As the code capability of the core key capabilities of the big model, how does it present the comprehensive capabilities of the big model such as understanding, logic, reasoning, generation, etc., and how should it objectively assess the code capability of the big model and promote the iterative progress of the code capability of the big model? Participating in the meeting to share the topic.
Meng Guozhu, deputy director of the Architecture and Security Research Office of the Institute of Information Engineering of the Chinese Academy of Sciences, said that the Cheng-style AI code security standard will build an evaluation system from the five major dimensions of code generation, code translation, code error correction, code completion, and code security, and build a dataset that meets the requirements of the test task for each capability item, and is designed to cover the three indexes of code completeness, functional completeness, and text similarity, the compile We designed three indicators covering code integrity, functionality and text similarity, and six secondary indicators covering compilation correctness, runtime correctness, runtime timeout, pass@k, sample pass rate and CodeBleu.
When talking about how to develop the code capability of big language modeling, Meng Guozhu suggested,, improving the quality of code corpus and expanding the code-related multimodal dataset. Second, strengthen the code security alignment algorithm. Third, expand the big model context window. Fourth, strengthen defect detection and security enhancement for large models.
In addition, Liu Wenchang from Beijing Zhipu Huazhang Technology Co., Ltd. and Yue Guanjie from National Engineering Research Center for Software Engineering, Peking University presented "CodeGeeX Intelligent Programming Assistant" and "Software Security Practices Based on Large Models" respectively. Theme Sharing. At the same time, delegates discussed generative AI code security from different perspectives and put forward many insightful suggestions.
The delegates discussed the issue of generative AI code security from different perspectives and put forward many insightful suggestions.
The AI Security Working Group will continue to shoulder the important responsibility of promoting the development of AI security, relying on the resources and advantages of the Lab, strengthening communication and cooperation with all parties, and jointly promoting the preparation and implementation of generative AI code security standards, so as to provide a strong guarantee for the safe, healthy, and sustainable development of AI technology.The AI Security Working Group will also always adhere to the concept of science and technology for the good to ensure that AI technology development is always people-oriented and serves the interests of society and the public.
Translated with DeepL.com (free version)