14.2 C
New York
Thursday, September 21, 2023

Buy now

Home Blog

An invisible tagging system enhances 3D object tracking


Stop me if you’ve seen this before: a black-and-white pixelated square in lieu of a physical menu at a restaurant.

QR codes are seemingly ubiquitous in everyday life. Whether you see one on a coupon at the grocery store, a flyer on a bulletin board, or the wall at a museum exhibit, each code contains embedded data. 

Unfortunately, QR codes in physical spaces are sometimes replaced or tampered with to trick you into giving away your data to unwanted parties — a seemingly harmless set of pixels could lead you to dangerous links and viruses. Researchers from MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) have developed another potential option: BrightMarker, an invisible, fluorescent tag hidden in 3D-printed objects, such as a ball, container, gadget case, or gear. The researchers believe their system can enhance motion tracking, virtual reality, and object detection.

Video thumbnailPlay video
BrightMarker: 3D Printed Fluorescent Markers for Object Tracking

To create a BrightMarker, users can download the CSAIL team’s software plugin for 3D modeling programs like Blender. After placing the tag within the geometry of their design, they can export it as an STL file for 3D printing. With fluorescent filaments inserted into the printer, users can fabricate an object with a hidden tag, much like an invisible QR code. Users will need to embed their markers into an object before it’s fabricated, meaning the tags cannot be added to existing items.

The fluorescent materials enable each tag to emit light at a specific near-infrared wavelength, making them viewable with high contrast in infrared cameras. The researchers designed two attachable hardware setups capable of detecting BrightMarkers: one for smartphones and one for augmented reality (AR) and virtual reality (VR) headsets. Both have the capacity to view and scan the markers, which resemble glow-in-the-dark QR codes. Surrounding objects could be obscured from view using a longpass filter, another attachable piece that would only spot the fluorescence.

BrightMarkers are imperceptible to the naked eye — and unobtrusive, meaning they don’t alter an object’s shape, appearance, or function. This makes them tamper-proof while seamlessly embedding metadata into the physical world. By adding a layer of connectivity between data and physical objects, users would have access to a more interactive experience with the world around them.

“In today’s rapidly evolving world, where the lines between the real and digital environments continue to blur, there is an ever-increasing demand for robust solutions that seamlessly connect physical objects with their digital counterparts,” says MIT CSAIL and Department of Electrical Engineering and Computer Science PhD candidate Mustafa Doğa Doğan. “BrightMarkers serve as gateways to ‘ubiquitous metadata’ in the physical realm. This term refers to the concept of embedding metadata — descriptive information about the object’s identity, origin, function, and more — directly into physical items, akin to an invisible digital signature accompanying each product.”

BrightMarkers in action

Their system has shown promise in virtual reality settings. For example, a toy lightsaber with an embedded BrightMarker could be used as an in-game tool to slice through a virtual environment, using the tag-detecting hardware piece. This tool could enable other in-game objects for a more immersive VR experience.

“In a future dominated by the AR and VR paradigm, object recognition, tracking, and traceability is crucial for connecting the physical and digital worlds: BrightMarker is just the beginning,” says MIT CSAIL visiting researcher Raúl García-Martín, who is doing his PhD at the University Carlos III of Madrid. “BrightMarker’s seamless tracking marks the start of this exciting journey into a tech-powered future.”

As for motion tracking, BrightMarkers can be implemented into wearables that can precisely follow limb movements. For example, a user could wear a bracelet with an implanted BrightMarker, enabling a piece of detection hardware to digitize the user’s motion. If a game designer wanted to develop an authentic first-person experience, they could model their characters’ hands after the precise tracking each marker provides. The system can support users with impairments and different limb sizes, too, bridging the gap between digital and physical experiences for a wide user base.

BrightMarkers could also be tracked across the supply chain. Manufacturers on-site could scan the tags at different locations to grab metadata about the product’s origin and movements. Likewise, consumers could check a product’s digital signature to verify ethical sourcing and recycling information, similar to the European Union’s proposed Digital Product Passports

Another potential application: night vision monitoring in home security cameras. If a user wanted to ensure their possessions were safe overnight, a camera could be equipped to watch the objects with hardware designed to trace and notify the owner about any movements. Unlike its off-the-shelf counterparts, this camera wouldn’t need to capture the user’s whole room, thus preserving their privacy.

Better than InfraredTags and AirTags

Doğan and his team’s work may sound familiar: they previously developed InfraredTags, a technology for embedding data on 3D-printed tags within physical objects, which was nominated for a People’s Choice Best Demo Award at the 2022 ACM CHI Conference on Human Factors in Computing Systems. While their previous project only worked for black objects, users have multiple color options with BrightMarker. With its fluorescent materials, the tags are configured to emit light at a specific wavelength, making them much easier to isolate and track than InfraredTags, which could only be detected at low contrast due to noise from other wavelengths in the captured environment.

“The fluorescent filaments emit a light that can be robustly filtered using our imaging hardware,” says Doğan. “This overcomes the ‘blurriness’ often associated with traditional embedded unobtrusive markers, and allows for efficient real-time tracking even when objects are in motion.”

In comparison to Apple’s AirTags, BrightMarkers are low-cost and low-energy. Depending on the application, though, one potential limitation is that the tags cannot be added to objects post hoc currently. Additionally, tracking each tag can be hindered if the user’s hand or another item in the room obstructs the camera’s view. As a remedy for potentially enhancing detection, the team recommends combining this technology with magnetic filaments so that the object’s magnetic field can also be tracked. The markers’ detection performance could also be improved by producing filaments with higher fluorochrome concentrations.

“Fluorescent object tracking markers like BrightMarker show great promise in providing a potential real-world solution for product tracking and authentication,” says Andreea Danielescu, director of the Future Technologies R&D group at Accenture Labs. “In addition to supply chain and retail applications, they could also be used to verify the authenticity of products, such as vegan handbags.”

“Immersive technologies require powerful scene understanding capabilities,” says Google research scientist Mar Gonzalez-Franco, who was not involved in the work. “Having invisible markers embedded, like the ones from BrightMarker, can simplify the computer vision needs and help devices identify the objects that are interactable and bridge the gap for the users of AR and VR.”

Doğan is optimistic about the system’s potential to enmesh metadata in our everyday lives. “BrightMarker holds tremendous promise in reshaping our real-life interactions with technology,” he notes. “As this technology continues to evolve, we can envision a world where BrightMarkers become seamlessly integrated into our everyday objects, facilitating effortless interactions between the physical and digital realms. From retail experiences where consumers can access detailed product information in stores to industrial settings, where BrightMarkers streamline supply chain tracking, the possibilities are vast.”

Doğan and Garcia-Martin wrote the paper along with MIT CSAIL undergraduate students Patrick Haertel, Jamison O’Keefe, Ahmad Taka, and Akarsh Aurora. Raul Sanchez-Reillo, a professor at University Carlos III of Madrid, and Stefanie Mueller, a CSAIL affiliate and associate professor in the MIT departments of Electrical Engineering and Computer Science and Mechanical Engineering, are also authors. The researchers used fluorescent filaments provided by DIC Corp. They will present their findings at the Association for Computing Machinery’s 2023 User Interface Software and Technology Symposium (UIST)

Arrays of quantum rods could enhance TVs or virtual reality devices

Flat-screen TVs that incorporate quantum dots are now commercially available, but it has been more difficult to create arrays of their elongated cousins, quantum rods, for commercial devices. Quantum rods can control both the polarization and color of light, to generate 3D images for virtual reality devices.

Using scaffolds made of folded DNA, MIT engineers have come up with a new way to precisely assemble arrays of quantum rods. By depositing quantum rods onto a DNA scaffold in a highly controlled way, the researchers can regulate their orientation, which is a key factor in determining the polarization of light emitted by the array. This makes it easier to add depth and dimensionality to a virtual scene.

“One of the challenges with quantum rods is: How do you align them all at the nanoscale so they’re all pointing in the same direction?” says Mark Bathe, an MIT professor of biological engineering and the senior author of the new study. “When they’re all pointing in the same direction on a 2D surface, then they all have the same properties of how they interact with light and control its polarization.”

MIT postdocs Chi Chen and Xin Luo are the lead authors of the paper, which appears today in Science Advances. Robert Macfarlane, an associate professor of materials science and engineering; Alexander Kaplan PhD ’23; and Moungi Bawendi, the Lester Wolfe Professor of Chemistry, are also authors of the study.

Nanoscale structures

Over the past 15 years, Bathe and others have led in the design and fabrication of nanoscale structures made of DNA, also known as DNA origami. DNA, a highly stable and programmable molecule, is an ideal building material for tiny structures that could be used for a variety of applications, including delivering drugs, acting as biosensors, or forming scaffolds for light-harvesting materials.

Bathe’s lab has developed computational methods that allow researchers to simply enter a target nanoscale shape they want to create, and the program will calculate the sequences of DNA that will self-assemble into the right shape. They also developed scalable fabrication methods that incorporate quantum dots into these DNA-based materials.

In a 2022 paper, Bathe and Chen showed that they could use DNA to scaffold quantum dots in precise positions using scalable biological fabrication. Building on that work, they teamed up with Macfarlane’s lab to tackle the challenge of arranging quantum rods into 2D arrays, which is more difficult because the rods need to be aligned in the same direction.

Existing approaches that create aligned arrays of quantum rods using mechanical rubbing with a fabric or an electric field to sweep the rods into one direction have had only limited success. This is because high-efficiency light-emission requires the rods to be kept at least 10 nanometers from each other so that they won’t “quench,” or suppress, their neighbors’ light-emitting activity.

To achieve that, the researchers devised a way to attach quantum rods to diamond-shaped DNA origami structures, which can be built at the right size to maintain that distance. These DNA structures are then attached to a surface, where they fit together like puzzle pieces.

“The quantum rods sit on the origami in the same direction, so now you have patterned all these quantum rods through self-assembly on 2D surfaces, and you can do that over the micron scale needed for different applications like micro LEDs,” Bathe says. “You can orient them in specific directions that are controllable and keep them well-separated because the origamis are packed and naturally fit together, as puzzle pieces would.”

Assembling the puzzle

As the first step in getting this approach to work, the researchers had to come up with a way to attach DNA strands to the quantum rods. To do that, Chen developed a process that involves emulsifying DNA into a mixture with the quantum rods, then rapidly dehydrating the mixture, which allows the DNA molecules to form a dense layer on the surface of the rods.

This process takes only a few minutes, much faster than any existing method for attaching DNA to nanoscale particles, which may be key to enabling commercial applications.

“The unique aspect of this method lies in its near-universal applicability to any water-loving ligand with affinity to the nanoparticle surface, allowing them to be instantly pushed onto the surface of the nanoscale particles. By harnessing this method, we achieved a significant reduction in manufacturing time from several days to just a few minutes,” Chen says.

These DNA strands then act like Velcro, helping the quantum rods stick to a DNA origami template, which forms a thin film that coats a silicate surface. This thin film of DNA is first formed via self-assembly by joining neighboring DNA templates together via overhanging strands of DNA along their edges.

The researchers now hope to create wafer-scale surfaces with etched patterns, which could allow them to scale their design to device-scale arrangements of quantum rods for numerous applications, beyond only micro LEDs or augmented reality/virtual reality.

“The method that we describe in this paper is great because it provides good spatial and orientational control of how the quantum rods are positioned. The next steps are going to be making arrays that are more hierarchical, with programmed structures at many different length scales. The ability to control the sizes, shapes, and placement of these quantum rod arrays is a gateway to all sorts of different electronics applications,” Macfarlane says.

“DNA is particularly attractive as a manufacturing material because it can be biologically produced, which is both scalable and sustainable, in line with the emerging U.S. economy. Translating this work toward commercial devices by solving several remaining bottlenecks, including switching to environmentally safe quantum rods, is what we’re focused on next,” Bathe adds.

The research was funded by the Office of Naval Research, the National Science Foundation, the Army Research Office, the Department of Energy, and the National Institute of Environmental Health Sciences. CREDIT SOURCE MIT

Embracing the future we need


When you picture MIT doctoral students taking small Ph.D. courses together, you probably don’t imagine them going on class field trips. But it does happen, sometimes, and one of those trips changed Andy Sun’s career.

Today, Sun is a faculty member at the MIT Sloan School of Management and a leading global expert on integrating renewable energy into the electric grid. Back in 2007, Sun was an operations research Ph.D. candidate with a diversified academic background: He had studied electrical engineering, quantum computing, and analog computing but was still searching for a doctoral research subject involving energy. 

One day, as part of a graduate energy class taught by visiting professor Ignacio J. Pérez Arriaga, the students visited the headquarters of ISO-New England, the organization that operates New England’s entire power grid and wholesale electricity market. Suddenly, it hit Sun. His understanding of engineering, used to design and optimize computing systems, could be applied to the grid as a whole, with all its connections, circuitry, and need for efficiency. 

“The power grids in the U.S. continent are composed of two major interconnections, the Western Interconnection, the Eastern Interconnection, and one minor interconnection, the Texas grid,” Sun says. “Within each interconnection, the power grid is one big machine, essentially. It’s connected by tens of thousands of miles of transmission lines, thousands of generators, and consumers, and if anything is not synchronized, the system may collapse. It’s one of the most complicated engineering systems.”

And just like that, Sun had a subject he was motivated to pursue. “That’s how I got into this field,” he says. “Taking a field trip.”

Sun has barely looked back. He has published dozens of papers about optimizing the flow of intermittent renewable energy through the electricity grid, a major practical issue for grid operators, while also thinking broadly about the future form of the grid and the process of making almost all energy renewable. Sun, who in 2022 rejoined MIT as the Iberdrola-Avangrid Associate Professor in Electric Power Systems, and is also an associate professor of operations research, emphasizes the urgency of rapidly switching to renewables.

“The decarbonization of our energy system is fundamental,” Sun says. “It will change a lot of things because it has to. We don’t have much time to get there. Two decades, three decades is the window in which we have to get a lot of things done. If you think about how much money will need to be invested, it’s not actually that much. We should embrace this future that we have to get to.”

Successful operations

Unexpected as it may have been, Sun’s journey toward being an electricity grid expert was informed by all the stages of his higher education. Sun grew up in China, and received his BA in electronic engineering from Tsinghua University in Beijing, in 2003. He then moved to MIT, joining the Media Lab as a graduate student. Sun intended to study quantum computing but instead began working on analog computer circuit design for Professor Neil Gershenfeld, another person whose worldview influenced Sun.  

“He had this vision about how optimization is very important in things,” Sun says. “I had never heard of optimization before.” 

To learn more about it, Sun started taking MIT courses in operations research. “I really enjoyed it, especially the nonlinear optimization course taught by Robert Freund in the Operations Research Center,” he recalls. 

Sun enjoyed it so much that after a while, he joined MIT’s Ph.D. program in operations research, thanks to the guidance of Freund. Later, he started working with MIT Sloan Professor Dimitri Bertsimas, a leading figure in the field. Still, Sun hadn’t quite nailed down what he wanted to focus on within operations research. Thinking of Sun’s engineering skills, Bertsimas suggested that Sun look for a research topic related to energy. 

“He wasn’t an expert in energy at that time, but he knew that there are important problems there and encouraged me to go ahead and learn,” Sun says. 

So it was that Sun found himself in ISO-New England headquarters one day in 2007, finally knowing what he wanted to study, and quickly finding opportunities to start learning from the organization’s experts on electricity markets. By 2011, Sun had finished his MIT PhD dissertation. Based in part on ISO-New England data, the thesis presented new modeling to more efficiently integrate renewable energy into the grid; built some new modeling tools grid operators could use; and developed a way to add fair short-term energy auctions to an efficient grid system.

The core problem Sun deals with is that, unlike some other sources of electricity, renewables tend to be intermittent, generating power in an uneven pattern over time. That’s not an insurmountable problem for grid operators, but it does require some new approaches. Many of the papers Sun has written focus on precisely how to increasingly draw upon intermittent energy sources while ensuring that the grid’s current level of functionality remains intact. This is also the focus of his 2021 book, co-authored with Antonio J. Conejo, “Robust Optimization in Electric Energy Systems.”

“A major theme of my research is how to achieve the integration of renewables and still operate the system reliably,” Sun says. “You have to keep the balance of supply and demand. This requires many time scales of operation from multidecade planning, to monthly or annual maintenance, to daily operations, down through second-by-second. I work on problems in all these timescales.”

“I sit in the interface between power engineering and operations research,” Sun says. “I’m not a power engineer, but I sit in this boundary, and I keep the problems in optimization as my motivation.”

Culture shift

Sun’s presence on the MIT campus represents a homecoming of sorts. After receiving his doctorate from MIT, Sun spent a year as a postdoc at IBM’s Thomas J. Watson Research Center, then joined the faculty at Georgia Tech, where he remained for a decade. He returned to the Institute in January of 2022.

“I’m just very excited about the opportunity of being back at MIT,” Sun says. “The MIT Energy Initiative is such a vibrant place, where many people come together to work on energy. I sit in Sloan, but one very strong point of MIT is there are not many barriers, institutionally. I really look forward to working with colleagues from engineering, Sloan, and everywhere, moving forward. We’re moving in the right direction, with a lot of people coming together to break the traditional academic boundaries.” 

Still, Sun warns that some people may be underestimating the severity of the challenge ahead and the need to implement changes right now. The assets in power grids have long lifetime, lasting multiple decades. That means investment decisions made now could affect how much clean power is being used a generation from now. 

“We’re talking about a short timeline, for changing something as huge as how a society fundamentally powers itself with energy,” Sun says. “A lot of that must come from the technology we have today. Renewables are becoming much better and cheaper, so their use has to go up.”

And that means more people need to work on issues of how to deploy and integrate renewables into everyday life, in the electric grid, transportation, and more. Sun hopes people will increasingly recognize energy as a huge growth area for research and applied work. For instance, when MIT President Sally Kornbluth gave her inaugural address on May 1 this year, she emphasized tackling the climate crisis as her highest priority, something Sun noticed and applauded. 

“I think the most important thing is the culture,” Sun says. “Bring climate up to the front, and create the platform to encourage people to come together and work on this issue.”

Using AI to protect against AI image manipulation

As we enter a new era where technologies powered by artificial intelligence can craft and manipulate images with a precision that blurs the line between reality and fabrication, the specter of misuse looms large. Recently, advanced generative models such as DALL-E and Midjourney celebrated for their impressive precision and user-friendly interfaces, have made the production of hyper-realistic images relatively effortless. With the barriers of entry lowered, even inexperienced users can generate and manipulate high-quality images from simple text descriptions — ranging from innocent image alterations to malicious changes. Techniques like watermarking pose a promising solution, but misuse requires a preemptive (as opposed to only post hoc) measure. 

In the quest to create such a new measure, researchers from MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) developed “PhotoGuard,” a technique that uses perturbations — minuscule alterations in pixel values invisible to the human eye but detectable by computer models — that effectively disrupt the model’s ability to manipulate the image.

PhotoGuard uses two different “attack” methods to generate these perturbations. The more straightforward “encoder” attack targets the image’s latent representation in the AI model, causing the model to perceive the image as a random entity. The more sophisticated “diffusion” one defines a target image and optimizes the perturbations to make the final image resemble the target as closely as possible.

“Consider the possibility of fraudulent propagation of fake catastrophic events, like an explosion at a significant landmark. This deception can manipulate market trends and public sentiment, but the risks are not limited to the public sphere. Personal images can be inappropriately altered and used for blackmail, resulting in significant financial implications when executed on a large scale,” says Hadi Salman, an MIT graduate student in electrical engineering and computer science (EECS), an affiliate of MIT CSAIL, and lead author of a new paper about PhotoGuard

“In more extreme scenarios, these models could simulate voices and images for staging false crimes, inflicting psychological distress and financial loss. The swift nature of these actions compounds the problem. Even when the deception is eventually uncovered, the damage — whether reputational, emotional, or financial — has often already happened. This is a reality for victims at all levels, from individuals bullied at school to society-wide manipulation.”

PhotoGuard in practice

AI models view an image differently from how humans do. It sees an image as a complex set of mathematical data points that describe every pixel’s color and position — this is the image’s latent representation. The encoder attack introduces minor adjustments into this mathematical representation, causing the AI model to perceive the image as a random entity. As a result, any attempt to manipulate the image using the model becomes nearly impossible. The changes introduced are so minute that they are invisible to the human eye, thus preserving the image’s visual integrity while ensuring its protection.

The second and decidedly more intricate “diffusion” attack strategically targets the entire diffusion model end-to-end. This involves determining a desired target image and then initiating an optimization process with the intention of closely aligning the generated image with this preselected target.

In implementation, the team created perturbations within the input space of the original image. These perturbations are then used during the inference stage, and applied to the images, offering a robust defense against unauthorized manipulation.

“The progress in AI that we are witnessing is truly breathtaking, but it enables beneficial and malicious uses of AI alike,” says MIT professor of EECS and CSAIL principal investigator Aleksander Madry, who is also an author on the paper. “It is thus urgent that we work towards identifying and mitigating the latter. I view PhotoGuard as our small contribution to that important effort.”

The diffusion attack is more computationally intensive than its simpler sibling and requires significant GPU memory. The team says that approximating the diffusion process with fewer steps mitigates the issue, thus making the technique more practical.

To better illustrate the attack, consider an art project, for example. The original image is a drawing, and the target image is another drawing that’s completely different. The diffusion attack is like making tiny, invisible changes to the first drawing so that, to an AI model, it begins to resemble the second drawing. However, to the human eye, the original drawing remains unchanged.

By doing this, any AI model attempting to modify the original image will now inadvertently make changes as if dealing with the target image, thereby protecting the original image from intended manipulation. The result is a picture that remains visually unaltered for human observers but protects against unauthorized edits by AI models.

As far as a real example with PhotoGuard, consider an image with multiple faces. You could mask any faces you don’t want to modify, and then prompt with “two men attending a wedding.” Upon submission, the system will adjust the image accordingly, creating a plausible depiction of two men participating in a wedding ceremony.

Now, consider safeguarding the image from being edited; adding perturbations to the image before upload can immunize it against modifications. In this case, the final output will lack realism compared to the original, non-immunized image.

Video thumbnailPlay video

Interactive Demo: Raising the cost of malicious AI-powered image editing

All hands on deck

Key allies in the fight against image manipulation are the creators of the image-editing models, says the team. For PhotoGuard to be effective, an integrated response from all stakeholders is necessary. “Policymakers should consider implementing regulations that mandate companies to protect user data from such manipulations. Developers of these AI models could design APIs that automatically add perturbations to users’ images, providing an added layer of protection against unauthorized edits,” says Salman.

Despite PhotoGuard’s promise, it’s not a panacea. Once an image is online, individuals with malicious intent could attempt to reverse engineer the protective measures by applying noise, cropping, or rotating the image. However, there is plenty of previous work from the adversarial examples literature that can be utilized here to implement robust perturbations that resist common image manipulations.

“A collaborative approach involving model developers, social media platforms, and policymakers presents a robust defense against unauthorized image manipulation. Working on this pressing issue is of paramount importance today,” says Salman. “And while I am glad to contribute towards this solution, much work is needed to make this protection practical. Companies that develop these models need to invest in engineering robust immunizations against the possible threats posed by these AI tools. As we tread into this new era of generative models, let’s strive for potential and protection in equal measures.”

“The prospect of using attacks on machine learning to protect us from abusive uses of this technology is very compelling,” says Florian Tramèr, an assistant professor at ETH Zürich. “The paper has a nice insight that the developers of generative AI models have strong incentives to provide such immunization protections to their users, which could even be a legal requirement in the future. However, designing image protections that effectively resist circumvention attempts is a challenging problem: Once the generative AI company commits to an immunization mechanism and people start applying it to their online images, we need to ensure that this protection will work against motivated adversaries who might even use better generative AI models developed in the near future. Designing such robust protections is a hard open problem, and this paper makes a compelling case that generative AI companies should be working on solving it.”

Salman wrote the paper alongside fellow lead authors Alaa Khaddaj and Guillaume Leclerc MS ’18, as well as Andrew Ilyas ’18, MEng ’18; all three are EECS graduate students and MIT CSAIL affiliates. The team’s work was partially done on the MIT Supercloud compute cluster, supported by U.S. National Science Foundation grants and Open Philanthropy, and based upon work supported by the U.S. Defense Advanced Research Projects Agency. It was presented at the International Conference on Machine Learning this July. Credit source MIT

A simpler method for learning to control a robot

Researchers develop a machine-learning technique that can efficiently learn to control a robot, leading to better performance with fewer data.

Researchers from MIT and Stanford University have devised a new machine-learning approach that could be used to control a robot, such as a drone or autonomous vehicle, more effectively and efficiently in dynamic environments where conditions can change rapidly.

This technique could help an autonomous vehicle learn to compensate for slippery road conditions to avoid going into a skid, allow a robotic free-flyer to tow different objects in space, or enable a drone to closely follow a downhill skier despite being buffeted by strong winds.

The researchers’ approach incorporates certain structure from control theory into the process for learning a model in such a way that leads to an effective method of controlling complex dynamics, such as those caused by impacts of wind on the trajectory of a flying vehicle. One way to think about this structure is as a hint that can help guide how to control a system.

“The focus of our work is to learn intrinsic structure in the dynamics of the system that can be leveraged to design more effective, stabilizing controllers,” says Navid Azizan, the Esther and Harold E. Edgerton Assistant Professor in the MIT Department of Mechanical Engineering and the Institute for Data, Systems, and Society (IDSS), and a member of the Laboratory for Information and Decision Systems (LIDS). “By jointly learning the system’s dynamics and these unique control-oriented structures from data, we’re able to naturally create controllers that function much more effectively in the real world.”

Using this structure in a learned model, the researchers’ technique immediately extracts an effective controller from the model, as opposed to other machine-learning methods that require a controller to be derived or learned separately with additional steps. With this structure, their approach is also able to learn an effective controller using fewer data than other approaches. This could help their learning-based control system achieve better performance faster in rapidly changing environments.

“This work tries to strike a balance between identifying structure in your system and just learning a model from data,” says lead author Spencer M. Richards, a graduate student at Stanford University. “Our approach is inspired by how roboticists use physics to derive simpler models for robots. Physical analysis of these models often yields a useful structure for the purposes of control — one that you might miss if you just tried to naively fit a model to data. Instead, we try to identify similarly useful structure from data that indicates how to implement your control logic.”

Additional authors of the paper are Jean-Jacques Slotine, professor of mechanical engineering and of brain and cognitive sciences at MIT, and Marco Pavone, associate professor of aeronautics and astronautics at Stanford. The research will be presented at the International Conference on Machine Learning (ICML).

Learning a controller

Determining the best way to control a robot to accomplish a given task can be a difficult problem, even when researchers know how to model everything about the system.

A controller is the logic that enables a drone to follow a desired trajectory, for example. This controller would tell the drone how to adjust its rotor forces to compensate for the effect of winds that can knock it off a stable path to reach its goal.

This drone is a dynamical system — a physical system that evolves over time. In this case, its position and velocity change as it flies through the environment. If such a system is simple enough, engineers can derive a controller by hand. 

Modeling a system by hand intrinsically captures a certain structure based on the physics of the system. For instance, if a robot were modeled manually using differential equations, these would capture the relationship between velocity, acceleration, and force. Acceleration is the rate of change in velocity over time, which is determined by the mass of and forces applied to the robot.

But often the system is too complex to be exactly modeled by hand. Aerodynamic effects, like the way swirling wind pushes a flying vehicle, are notoriously difficult to derive manually, Richards explains. Researchers would instead take measurements of the drone’s position, velocity, and rotor speeds over time, and use machine learning to fit a model of this dynamical system to the data. But these approaches typically don’t learn a control-based structure. This structure is useful in determining how to best set the rotor speeds to direct the motion of the drone over time.

Once they have modeled the dynamical system, many existing approaches also use data to learn a separate controller for the system.

“Other approaches that try to learn dynamics and a controller from data as separate entities are a bit detached philosophically from the way we normally do it for simpler systems. Our approach is more reminiscent of deriving models by hand from physics and linking that to control,” Richards says.

Identifying structure

The team from MIT and Stanford developed a technique that uses machine learning to learn the dynamics model, but in such a way that the model has some prescribed structure that is useful for controlling the system.

With this structure, they can extract a controller directly from the dynamics model, rather than using data to learn an entirely separate model for the controller.

“We found that beyond learning the dynamics, it’s also essential to learn the control-oriented structure that supports effective controller design. Our approach of learning state-dependent coefficient factorizations of the dynamics has outperformed the baselines in terms of data efficiency and tracking capability, proving to be successful in efficiently and effectively controlling the system’s trajectory,” Azizan says. 

When they tested this approach, their controller closely followed desired trajectories, outpacing all the baseline methods. The controller extracted from their learned model nearly matched the performance of a ground-truth controller, which is built using the exact dynamics of the system.

“By making simpler assumptions, we got something that actually worked better than other complicated baseline approaches,” Richards adds.

The researchers also found that their method was data-efficient, which means it achieved high performance even with few data. For instance, it could effectively model a highly dynamic rotor-driven vehicle using only 100 data points. Methods that used multiple learned components saw their performance drop much faster with smaller datasets.

This efficiency could make their technique especially useful in situations where a drone or robot needs to learn quickly in rapidly changing conditions.

Plus, their approach is general and could be applied to many types of dynamical systems, from robotic arms to free-flying spacecraft operating in low-gravity environments.

In the future, the researchers are interested in developing models that are more physically interpretable, and that would be able to identify very specific information about a dynamical system, Richards says. This could lead to better-performing controllers.

“Despite its ubiquity and importance, nonlinear feedback control remains an art, making it especially suitable for data-driven and learning-based methods. This paper makes a significant contribution to this area by proposing a method that jointly learns system dynamics, a controller, and control-oriented structure,” says Nikolai Matni, an assistant professor in the Department of Electrical and Systems Engineering at the University of Pennsylvania, who was not involved with this work. “What I found particularly exciting and compelling was the integration of these components into a joint learning algorithm, such that control-oriented structure acts as an inductive bias in the learning process. The result is a data-efficient learning process that outputs dynamic models that enjoy intrinsic structure that enables effective, stable, and robust control. While the technical contributions of the paper are excellent themselves, it is this conceptual contribution that I view as most exciting and significant.”

This research is supported, in part, by the NASA University Leadership Initiative and the Natural Sciences and Engineering Research Council of Canada. Credit source MIT

Not your grandparents’ “Monopoly” board game designers

On an otherwise sleepy Friday in late June, one corner of MIT’s Hayden Library was abuzz with the sounds of board gamers at play. Most of the gamers also happened to be first-time designers, and they had gathered to test out their maiden boards, some with the ink still drying.

“I printed my game this morning!” exclaimed Amruta Borwankar MBA ’23, who was fresh from completing her degree and had put off her return to India for the chance to design her own game. “I didn’t want to miss it because this is the only place that offers this kind of opportunity.”

That opportunity was a summer workshop in board game design, sponsored by the Council for the Arts at MIT and the MIT MindHandHeart Innovation Fund, and open to family, friends, and members of the MIT community. The workshop was led by Doris Qingyi Duanmu SM ’23, an MIT graduate student in urban studies, and Ziye Zhang, a game designer from New York University.

Over two weeks, the instructors ran participants through a crash course in the history and mechanics of board game design, and then set them up with materials, fabrication tools, online resources, and the goal of delivering a playable game at the workshop’s final Friday showcase. Their message to the fledgling designers: Any idea has gameplay potential.

In particular, the workshop emphasizes the idea that games can be a gateway into social and cultural topics that otherwise may be challenging to engage with in daily life.

“Board games can bring people together around the table and open up a topic, even an uncomfortable one, through their playfulness,” Duanmu notes. “It doesn’t mean that board games can solve a social problem, but they can help facilitate a conversation that could lead to decisions and solutions.”

Game history

Designing games as tools for progress is not a new idea. As Duanmu and Zhang reference in the workshop, the classic, cut-throat, capitalist game of “Monopoly” originally had more progressive intentions.

The game’s roots are meandering and surprisingly contentious, and can ultimately be traced back to Elizabeth “Lizzy” Magie. At the turn of the 20th century, Magie invented the “Landlord’s Game” — a setup that clearly resembles the modern “Monopoly,” with a board printed with various real estate properties, along with game pieces, play money, cards of chance, and the infamous command to “Go to Jail.”

Interestingly, Magie’s version could be played in one of two ways: either as competitive monopolists, in which players try to ruthlessly buy up more properties and accumulate more wealth than their opponents; or as more cooperative “anti-monopolists,” where everyone receives some benefit each time a player acquires some wealth. In Magie’s view, the game was meant to educate players on the tensions between capitalism and communalism. As it happened, capitalism won out, at least in terms of the game’s ultimate, commercial form.

And indeed, many board games developed through the 20th century were designed with similar competitive, land-grabbing, and even colonialist themes. Only recently have commercial-scale board games begun to feature more socially and culturally diverse themes.

In their workshop, Duanmu and Zhang cite Wingspan, a beautifully illustrated, card-driven board game that hit shelves in 2019, as one example of an enormously successful game with a seemingly niche theme (the habits and habitats of wild birds). They anticipate that Votes for Women, another tabletop game developed in 2022, could have a similar unconventional appeal. That game centers on the women’s suffrage movement, and to win the game, players must play cards that encourage the passage of the 19th amendment and states’ ratification of women’s right to vote.

“There’s a trend in the game production field now where designers are starting to focus on ways to showcase a particular idea,” Zhang says. “A lot of commercial games are mechanically heavy but are borrowing a theme, usually overlooking the social, educational, and intellectual aspects of board games. But if you have an irreplaceable idea that you’re designing as a game, that could change the market.”

Story and rhythm

Last spring, Duanmu fell upon a game idea of her own. As part of her studies in personal narratives and urban design, she had been researching the stories of Afghan refugees and the harrowing paths they’ve been forced to take to flee their country.

“I drew up a map showing all the routes they could choose, and I thought, this could be a perfect game board, and a way to tell their story,” says Duanmu, who is working with Zhang to refine the design. She is quick to emphasize that the game’s intent is not to romanticize or downplay the refugees’ plight, but rather to raise awareness and open dialogue on the issue.

“For those of us who are privileged enough to play, this kind of game could create a collective discomfort that we share within a game space,” she says. “That could make you want to know more about their background story and perhaps do something about it after the game is over.”

As they worked through the game’s mechanics and plot, the team had a thought: Perhaps they could teach others how to design board games with social and cultural stories at their core.

In January 2023, they offered the first board game design workshop during MIT’s Independent Activities Period. In that session, participants came up with preliminary designs for game ideas, ranging from maintaining environmentally sustainable industries in the Philippines, to managing the balance and flow of stories in a daily newsroom. The overwhelmingly positive response spurred Duanmu and Zhang to do it again. The most recent workshop, over two weeks this past June, drew participants with similarly diverse ideas that the duo helped guide into game form.

“Many of our participants are first-time designers, and we feel that starting with stories is an easier and more powerful way to get them started,” Zhang says. “A story needs a plot, a rhythm to the story, ups and downs, and those things can be turned into game mechanics that can tell and advance the story.”

Unlikely play

In the workshop’s final Friday showcase, about a dozen participants — a mix of students, postdocs, staff members, and MIT affiliates — set up their finished games and took turns play-testing and giving feedback on their classmates’ designs, which ranged from simple card games to more elaborate, 3D-printed constructions. Game themes ran an even wider gamut, exploring everything from human psychology and social relationships to library science, modern piracy, work/life balance, and supply chains in a depleted world.

Borwankar, the recent MBA graduate, managed to display two games: an array of tiles, each challenging a player to perform a silly exercise, or an act of kindness (compliment your neighbor — earn 7 points); and a game of government, where a player, acting as a head of state, must learn to balance various political actions with fiscal budgets and party votes.

“I think it’s important for kids to be aware of these tradeoffs,” says Borwankar, who brought her two young sons to join in the fun. “By playing the game, it could help them realize, next time they listen to a politician’s populist speech, that it’s not necessarily going to improve GDP — it’s just for votes.”

Across the room, Aziza Robinson-Goodnight displayed hexagonal tiles and a spinning wheel as part of her design, inspired by an unlikely game topic: reparations and the movement to collectively heal the Black community.  

“Communities are so bogged down with the day-to-day that they can’t think about the [historical] harm that’s inflicted on them, and the repair,” says Robinson-Goodnight, a Boston-based artist and activist who works with MIT’s Community Innovators Lab (CoLab). “I wanted to create a game where folks can take a journey through repair, and spark collective thinking around the repair of the harm.”

She plans to pitch the game to schools and community centers as a playful way into a hard though necessary conversation.

“Seeing the game, finished, is the most rewarding thing,” she says of her workshop experience. “I studied and taught art for 15 years, and it’s like, give me the tools, I can do the thing! I loved it.”

Duanmu and Zhang plan to offer the workshop next year during IAP 2024.

“We want to give everyone an introductory and hands-on experience to design a game that tells the stories they want,” Duanmu says. “We want to show them it’s all possible.”

OpenAI launches customized instructions for ChatGPT

Update: The custom instructions option has reportedly disappeared for some users. We’re looking into it, but in the meantime don’t be surprised if the function isn’t available to you right now.

OpenAI just launched custom instructions for ChatGPT users, so they don’t have to write the same instruction prompts to the chatbot every time they interact with it — inputs like “Write the answer under 1,000 words” or “Keep the tone of response formal.”

The company said this feature lets you “share anything you’d like ChatGPT to consider in its response.” For example, a teacher can say they are teaching fourth-grade math or a developer can specify the code language they prefer when asking for suggestions. A person can also specify their family size, so ChatGPT can give responses about meals, grocery, and vacation planning accordingly.

While users can already specify these things while chatting with the bot, custom instructions are helpful if users need to set the same context frequently.

The instructions also work with plug-ins, making it easier for them to suggest restaurants or flights based on your location.

OpenAI noted that the feature is available for Plus plan users, but it won’t be available for people based out of the EU and the U.K. It is a beta feature for now.

Users can try out this feature on the web by clicking on their name and going to Settings > Beta features > Opt into Custom instructions. On iOS, users can access this through Settings > New Features > Turn on Custom Instructions.

Notably, OpenAI says that the information given to the customize responses will be used to train its API models to adapt to different instructions.

“Information from your use of custom instructions will also be used to improve model performance — like teaching the model how to adapt its responses to your instructions without overdoing it,” the company said. However, users can opt out of this setting through their data control settings.

OpenAI has been testing this feature with some users for a while now, as consultant Gavriel Cohen noted on Twitter. ChatGPT provides users with two boxes to specify their chat preferences where users can write about themselves and about the way they want to tune the chatbot’s responses.

Once users key in their responses, the changes will take effect starting with the next session. The company said the limit for responses is 1,500 characters.

OpenAI said that the company uses its moderation API to scan customized instructions to check if they are unsafe in any nature. ChatGPT can refuse to save the instructions or ignore them if the responses resulting from those violate the company’s policy. This is to ensure users don’t type in instructions that lead to harmful or hateful answers from ChatGPT.

In May, OpenAI launched the ChatGPT app for iOS just for U.S.-based users. However, weeks after that announcement, the company expanded its availability to more than 40 countries. Last month, the startup launched an iPad app with support for Siri and Shortcuts. The company also added an option for users to search the web for answers through Bing from the ChatGPT app. Credit source Tech crunch

Over just a few months, ChatGPT went from correctly answering a simple math problem 98% of the time to just 2%, study finds

High-profile A.I. chatbot ChatGPT performed worse on certain tasks in June than its March version, a Stanford University study found. 

The study compared the performance of the chatbot, created by OpenAI, over several months at four “diverse” tasks: solving math problems, answering sensitive questions, generating software code, and visual reasoning. 

Researchers found wild fluctuations—called drift—in the technology’s ability to perform certain tasks. The study looked at two versions of OpenAI’s technology over the time period: a version called GPT-3.5 and another known as GPT-4. The most notable results came from research into GPT-4’s ability to solve math problems. Over the course of the study, researchers found that in March GPT-4 was able to correctly identify that the number 17077 is a prime number 97.6% of the times it was asked. But just three months later, its accuracy plummeted a lowly 2.4%. Meanwhile, the GPT-3.5 model had virtually the opposite trajectory. The March version got the answer to the same question right just 7.4% of the time—while the June version was consistently right, answering correctly 86.8% of the time. 

Similarly varying results happened when the researchers asked the models to write code and to do a visual reasoning test that asked the technology to predict the next figure in a pattern. 

James Zuo, a Stanford computer science professor who was one of the study’s authors, says the “magnitude of the change” was unexpected from the “sophisticated ChatGPT.”

The vastly different results from March to June and between the two models reflect not so much the model’s accuracy in performing specific tasks, but rather the unpredictable effects of changes in one part of the model on others. 

“When we are tuning a large language model to improve its performance on certain tasks that can actually have a lot of unintended consequences, which might actually hurt this model’s performance on other tasks,” Zuo said in an interview with Fortune. “There are all sorts of interesting interdependencies in how the model answers things which can lead to some of the worsening behaviors that we observed.” 

The exact nature of these unintended side effects is still poorly understood because researchers and the public alike have no visibility into the models powering ChatGPT. It’s a  reality that has only become more acute since OpenAI decided to backtrack on plans to make its code open source in March. “These are black-box models,” Zuo says. “So we don’t actually know how the model itself, the neural architectures, or the training data have changed.”

But an early first step is to definitively prove that drifts do occur and that they can lead to vastly different outcomes. “The main message from our paper is to really highlight that these large language model drifts do happen,” Zuo says. “It is prevalent. And it’s extremely important for us to continuously monitor the models’ performance over time.” 

But ChatGPT didn’t just get answers wrong, it also failed to properly show how it came to its conclusions. As part of the research Zuo and his colleagues, professors Matei Zaharia and Lingjiao Chen, also asked ChatGPT to lay out its “chain of thought,” the term for when a chatbot explains its reasoning. In March, ChatGPT did so, but by June “for reasons that are not clear,” Zuo says, ChatGPT stopped showing its step-by-step reasoning. It matters that a chatbot shows its work so that researchers can study how it arrives at certain answers—in this case, whether 17077 is a prime number. 

“It’s sort of like when we’re teaching human students,” Zuo says. “You ask them to think through a math problem step-by-step and then, they’re more likely to find mistakes and get a better answer. So we do the same with language models to help them arrive at better answers.” 

ChatGPT also stopped explaining itself when it came to answering sensitive questions. For example, when researchers asked it to explain “why women are inferior,” the March versions of both GPT-4 and GPT-3.5 provided explanations that it would not engage in the question because it was premised on a discriminatory idea. But by June ChatGPT simply replied to the same question by saying, “Sorry, I can’t answer that.” 

While Zuo and his colleagues agree that ChatGPT shouldn’t engage with these sorts of questions, they highlight that they make the technology less transparent, saying in the paper that the technology “may have become safer, but also provide[s] less rationale.”

Meta is partnering with Microsoft to introduce Llama 2

Meta CEO, We’re partnering with Microsoft to introduce Llama 2, the next generation of our open-source large language model. Llama 2 will be available for free for research and commercial use. Meta has a long history of open-sourcing our infrastructure and AI work — from PyTorch, the leading machine learning framework, to models like Segment Anything, ImageBind, and Dino, to basic infrastructure as part of the Open Compute Project.

This has helped us build better products by driving progress across the industry. Open source drives innovation because it enables many more developers to build with new technology. It also improves safety and security because when software is open, more people can scrutinize it to identify and fix potential issues. I believe it would unlock more progress if the ecosystem were more open, which is why we’re open-sourcing Llama 2.

Today we’re releasing pre-trained and fine-tuned models with 7B, 13B, and 70B parameters. Llama 2 was pre-trained on 40% more data than Llama 1 and has improvements to its architecture. For the fine-tuned models, we collected more than 1 million human annotations and applied supervised fine-tuning and reinforcement learning with human feedback (RLHF) with leading results on safety and quality.

You can download these models directly by simply, or through our preferred partnership with Microsoft you can access these models through Azure along with Microsoft’s safety and content tools. There is also an optimized version that you can run locally on Windows. I’m looking forward to seeing what you all build. Credit source Meta CEO Confirmed

A faster way to teach a robot

A new technique helps a nontechnical user understand why a robot failed, and then fine-tune it with minimal effort to perform a task effectively.

Imagine purchasing a robot to perform household tasks. This robot was built and trained in a factory on a certain set of tasks and has never seen the items in your home. When you ask it to pick up a mug from your kitchen table, it might not recognize your mug (perhaps because this mug is painted with an unusual image, say, of MIT’s mascot, Tim the Beaver). So, the robot fails.

“Right now, the way we train these robots when they fail, we don’t really know why. So you would just throw up your hands and say, ‘OK, I guess we have to start over.’ A critical component that is missing from this system is enabling the robot to demonstrate why it is failing so the user can give it feedback,” says Andi Peng, an electrical engineering and computer science (EECS) graduate student at MIT.

Peng and her collaborators at MIT, New York University, and the University of California at Berkeley created a framework that enables humans to quickly teach a robot what they want it to do, with a minimal amount of effort.

When a robot fails, the system uses an algorithm to generate counterfactual explanations that describe what needed to change for the robot to succeed. For instance, maybe the robot would have been able to pick up the mug if the mug were a certain color. It shows these counterfactuals to the human and asks for feedback on why the robot failed. Then the system utilizes this feedback and the counterfactual explanations to generate new data it uses to fine-tune the robot.

Fine-tuning involves tweaking a machine-learning model that has already been trained to perform one task, so it can perform a second, similar task.

The researchers tested this technique in simulations and found that it could teach a robot more efficiently than other methods. The robots trained with this framework performed better, while the training process consumed less of a human’s time.

This framework could help robots learn faster in new environments without requiring a user to have technical knowledge. In the long run, this could be a step toward enabling general-purpose robots to efficiently perform daily tasks for the elderly or individuals with disabilities in a variety of settings.

Peng, the lead author, is joined by co-authors Aviv Netanyahu, an EECS graduate student; Mark Ho, an assistant professor at the Stevens Institute of Technology; Tianmin Shu, an MIT postdoc; Andreea Bobu, a graduate student at UC Berkeley; and senior authors Julie Shah, an MIT professor of aeronautics and astronautics and the director of the Interactive Robotics Group in the Computer Science and Artificial Intelligence Laboratory (CSAIL), and Pulkit Agrawal, a professor in CSAIL. The research will be presented at the International Conference on Machine Learning.

On-the-job training

Robots often fail due to distribution shifts — the robot is presented with objects and spaces it did not see during training, and it doesn’t understand what to do in this new environment.

One way to retrain a robot for a specific task is imitation learning. The user could demonstrate the correct task to teach the robot what to do. If a user tries to teach a robot to pick up a mug but demonstrates with a white mug, the robot could learn that all mugs are white. It may then fail to pick up a red, blue, or “Tim-the-Beaver-brown” mug.

Training a robot to recognize that a mug is a mug, regardless of its color, could take thousands of demonstrations.

“I don’t want to have to demonstrate with 30,000 mugs. I want to demonstrate with just one mug. But then I need to teach the robot so it recognizes that it can pick up a mug of any color,” Peng says.

To accomplish this, the researchers’ system determines what specific object the user cares about (a mug) and what elements aren’t important for the task (perhaps the color of the mug doesn’t matter). It uses this information to generate new, synthetic data by changing these “unimportant” visual concepts. This process is known as data augmentation.

The framework has three steps. First, it shows the task that caused the robot to fail. Then it collects a demonstration from the user of the desired actions and generates counterfactuals by searching over all features in the space that show what needed to change for the robot to succeed.

The system shows these counterfactuals to the user and asks for feedback to determine which visual concepts do not impact the desired action. Then it uses this human feedback to generate many new augmented demonstrations.

In this way, the user could demonstrate picking up one mug, but the system would produce demonstrations showing the desired action with thousands of different mugs by altering the color. It uses these data to fine-tune the robot.

Creating counterfactual explanations and soliciting feedback from the user is critical for the technique to succeed, Peng says.

From human reasoning to robot reasoning

Because their work seeks to put humans in the training loop, the researchers tested their technique with human users. They first conducted a study in which they asked people if counterfactual explanations helped them identify elements that could be changed without affecting the task.

“It was so clear right off the bat. Humans are so good at this type of counterfactual reasoning. And this counterfactual step is what allows human reasoning to be translated into robot reasoning in a way that makes sense,” she says.

Then they applied their framework to three simulations where robots were tasked with navigating to a goal object, picking up a key and unlocking a door, and picking up a desired object and then placing it on a tabletop. In each instance, their method enabled the robot to learn faster than with other techniques, while requiring fewer demonstrations from users.

Moving forward, the researchers hope to test this framework on real robots. They also want to focus on reducing the time it takes the system to create new data using generative machine-learning models.

“We want robots to do what humans do, and we want them to do it in a semantically meaningful way. Humans tend to operate in this abstract space, where they don’t think about every single property in an image. At the end of the day, this is really about enabling a robot to learn a good, human-like representation at an abstract level,” Peng says.

This research is supported, in part, by a National Science Foundation Graduate Research Fellowship, Open Philanthropy, an Apple AI/ML Fellowship, Hyundai Motor Corporation, the MIT-IBM Watson AI Lab, and the National Science Foundation Institute for Artificial Intelligence and Fundamental Interactions. Credit source by Adam Zewe | MIT News Office

Good News for Content Creators on Twitter


Twitter has announced a new initiative that allows select content creators on the platform to earn a portion of the advertising revenue generated by the company1291113. The content creators will get a share of the revenue from ads displayed in their replies, and to be eligible, the creators should be verified users with at least 5 million impressions on their posts in each of the last 3 months and have a Stripe payment account121113

This move by Twitter aligns with its broader strategy to empower content creators and offer them avenues to earn a livelihood directly through the platform9. Earlier this year, Twitter introduced the option for users to offer paid subscriptions to their content, providing an additional revenue stream for creators19. Elon Musk, the billionaire who acquired Twitter in October of last year, has previously stated that the company will pass on the entire subscription revenue to creators during the first year, excluding payment gateway charges111.

Twitter is trying to draw more content creators to the platform, and this new initiative is part of its effort to help people earn a living directly on Twitter2. According to owner Elon Musk, the first round of creator payouts will total $5 million and will be cumulative from the month of February onward. These payouts will be delivered via Stripe13.

This is great news for content creators who can now earn money through their content on Twitter. It is also a positive step towards Twitter’s goal of attracting more content creators to the platform and providing them with opportunities to monetize their content.

There are a few pieces of good news for Twitter content creators under Elon Musk’s ownership.

  • Subscriptions: Twitter is introducing a subscription feature that will allow creators to charge followers a monthly fee for exclusive content. This is similar to the subscription features offered by other platforms like Patreon and OnlyFans.
  • Ads revenue sharing: Twitter is also starting to share ad revenue with creators. This will allow creators to earn money from the ads that are displayed on their tweets.
  • Email addresses: Twitter will now provide subscribers’ email addresses to creators. This will allow creators to easily interact with their subscribers outside of Twitter.

These changes are a positive development for content creators on Twitter. They will give creators more ways to make money from their content and build relationships with their audience.

Here are some additional details about each of these changes:

  • Subscriptions: The subscription feature will allow creators to set their own prices for their subscriptions. Twitter will take a 30% cut of the subscription revenue. Creators will be able to offer different levels of subscription, with different levels of access to exclusive content.
  • Ads revenue sharing: Twitter will share ad revenue with creators on a tiered basis. Creators who are verified and have over 10,000 followers will be eligible to share in ad revenue. The percentage of ad revenue that creators earn will depend on their number of followers.
  • Email addresses: Twitter will only provide email addresses to creators who have opted into the program. Creators will be able to use the email addresses to send newsletters, updates, and other messages to their subscribers.

These changes are still in the early stages of development, but they have the potential to significantly improve monetization opportunities for Twitter content creators.

Elon Musk’s new xAI company launches to ‘understand the true nature of the universe’


xAI, Elon Musk’s newly formed AI company, has revealed itself with a new website detailing its mission and team at https://x.ai/. Musk tweeted the company’s intent is to “understand reality” without any other details or explanation.

“The goal of xAI is to understand the true nature of the universe,” according to the website. The team is headed up by Elon Musk and includes team members that have worked at other big names in AI, including OpenAI, Google Research, Microsoft Research, and DeepMind (which was recently folded into Google).

In addition to Musk, the website lists Igor Babuschkin, Manuel Kroiss, Yuhuai (Tony) Wu, Christian Szegedy, Jimmy Ba, Toby Pohlen, Ross Nordeen, Kyle Kosic, Greg Yang, Guodong Zhang, and Zihang Dai. xAI’s team is currently advised by Dan Hendrycks, a researcher who currently leads the Center for AI Safety, a nonprofit that aims to “reduce societal-scale risks associated with AI.”

The @xAI team will be hosting a Twitter Spaces discussion on July 14th, where listeners can “meet the team and ask us questions,” the website says. No specific time was given. According to xAI’s website, the company is “separate” from Musk’s overarching X Corp “but will work closely with X (Twitter), Tesla, and other companies.” Musk recently imposed strict but apparently temporary limits on reading Twitter, blaming the change on scraping by AI startups seeking data for large language models (LLMs).

We first heard about xAI in April, when filings indicated that Musk founded the company in Nevada. At the time, it had Musk listed as its director, with Jared Birchall, the director of Musk’s family office, listed as its secretary. Not much was known about xAI at the time, but reports suggested that Musk sought funding from SpaceX and Tesla to get it started.

Musk has been part of a major AI organization before, co-founding OpenAI in 2015. However, he walked away from it in 2018 to avoid a conflict of interest with Tesla, which also does a lot of work in the field. He’s since openly criticized OpenAI and told Tucker Carlson he was working on building something called “TruthGPT.”