• 11 Posts
  • 21 Comments
Joined 1 year ago
cake
Cake day: April 4th, 2024

help-circle











  • It did depend a little bit, what kind of machine/production line he was working on. Before he retired, he worked for an automation engineering company and had different projects in other EU countries, and tried to be understandable for people in those places. He once even coded some Siemens control panel for an aluminum oven loading robot in the czech republic and tried to translate everything to czech with a dictionary (to have the panel info available in czech,english and German). He did of course speak to the foreman of the workers to get it correct.










  • Depends on what you do with it. Synthetic data seems to be really powerful if it’s human controlled and well built. Stuff like tiny stories (simple llm-generated stories that only use the complexity of a 3-year olds vocabulary) can be used to make tiny language models produce sensible English output. My favourite newer example is the base data for AlphaProof (llm-generated translations of proofs in Math-Papers to the proof-validation system LEAN) to teach an LLM the basic structure of Mathematics proofs. The validation in LEAN itself can be used to only keep high-quality (i.e. correct) proofs. Since AlphaProof is basically a reinforcement learning routine that uses an llm to generate good ideas for proof steps to reduce the size of the space of proof steps, applying it yields new correct proofs that can be used to further improve its internal training data.