• Hey, guest user. Hope you're enjoying NeoGAF! Have you considered registering for an account? Come join us and add your take to the daily discourse.

Ninty files patent for DLSS 2.0 proprietary 4K technology

Marty-McFly

Marty-McFly

Banned
The 4K future for Switch Advance looks inevitable (BS rumors from Bloomberg aside).

Interesting Nintendo is choosing to go with their own DLSS solution.


What year this comes out who knows. I'm still definitely all over OLED because who knows when we'll actually see this?

Will it be an extension of the current Switch or is this Switch 2?

and how will this effect its surging third party support?

patents for "SYSTEMS AND METHODS FOR MACHINE LEARNED IMAGE CONVERSION". AI upscaling akin to Nvidia's DLSS and Intel's XeSS.

patents were filed on March 25, 2020 and published today. the inventors are all listed from France, patents mention the methods can be accelerated by matrix multiplication hardware, like Nvidia's Tensor Cores



FAjrNlBX0AULp3W



patents are viewable here

SYSTEMS AND METHODS FOR MACHINE LEARNED IMAGE CONVERSION - NINTENDO CO., LTD.​

A computer system is provided for converting images through use of a trained neural network. A source image is divided into blocks and context data is added to each pixel block. The context blocks are
www.freepatentsonline.com
www.freepatentsonline.com

SYSTEMS AND METHODS FOR MACHINE LEARNED IMAGE CONVERSION - NINTENDO CO., LTD.​

A computer system is provided for converting images through use of a trained neural network. A source image is divided into blocks and context data is added to each pixel block. The context blocks are
www.freepatentsonline.com
www.freepatentsonline.com
Click to expand...

3EccFja.gif
 
Last edited:
martino

martino

Member
Dunno lot of uncertainties at the moment...
Will games push more cpu for something not graphics related at some point ?
Will ml or shader usage explode for multiple game usages not only graphics related too ?
If games pushes one of the two dlss will not some anything for them for a 10 to 20 w apu.
 
Last edited:
M

MrSec84

Member
Banjo64 said:
Switch 2 for sure, this will be years away though.
Click to expand...
I'm doubtful of this, given that even though the Wii was a crazy success for Nintendo they still launched the Wii U 6 years after that platform, the tech on the Switch is well behind the PS4 or XB1, let alone the PS5 and XBox Series consoles. Nintendo now has a taste for the 3rd party sales it has gotten on Switch, seems like they'll release a new platform with a similar hybrid style to Switch late next year or sometime in 2023.

Valve has shown that the hardware exists to be able to at least run a baseline level of hardware capable of running any current multiplats and that's using hardware that doesn't have DLSS level capabilities yet, NVidia just has the tech already.
Best thing is that DLSS can be used on low power mobile devices, the performance mode of DLSS 2.0 or higher works great and there's no reason why Nintendo can't have developers coding those features into their games from day one of game development.

By this time next year Tensor hardware is going to be incredibly mature, same goes for Ray Tracing, chip shortages should also be much improved, we likely would have had at least a pro version of the Switch this year without the manufacturing issues that have plagued the world during the Pandemic.
 
Barakov

Barakov

Member
Marty-McFly said:
The 4K future for Switch Advance looks inevitable (BS rumors from Bloomberg aside).

Interesting Nintendo is choosing to go with their own DLSS solution.


What year this comes out who knows. I'm still definitely all over OLED because who knows when we'll actually see this?

Will it be an extension of the current Switch or is this Switch 2?

and how will this effect its surging third party support?



3EccFja.gif
Click to expand...
Iq464af.gif
 
GametimeUK

GametimeUK

Member
lh032 said:
Part of me wish they release a switch 2 in 2022, that would create a lot of drama among the oled buyers xD
Click to expand...
It wouldn't be the first time Nintendo released an updated console and then a successor the following year if I remember correctly. I believe the DSi XL released in 2010 (outside of Japan) and the 3DS released in 2011. Unless someone can correct me or provide another example of when they've done this.
 
Marvel14

Marvel14

Banned
Coolwhhip said:
Nintendo creating their own DLSS is like a monkey designing a space ship.

Why would they unless they drop nvidia?
Click to expand...

My thoughts exactly. Is this their way of letting their customers know it's coming?

Haven't seen anyone mention of this but Nintendo now seems to bring out their new box a couple of years after MS/PS 's latest. This ensures no rapid copying by its competitors (as with motion control) and a cheaper, higher power offering than they would otherwise be able to offer, that balances cost and price point. So 2022-2023 is definitely the time for their next console.


I don't see them having any expertise at all with DLSS and they lose all the benefits of a wider industry standard and all the upgrade benefits that would bring.....

This is either more fresh bullshit or a comms stunt.
 
Last edited:
Coolwhhip

Coolwhhip

Neophyte
Marty-McFly said:
They are still likely working with Nvidia.

Saying that godfathers of gaming couldn't create well designed high tech hardware if they wanted to is ridiculous.

You must have missed the Gamecube and N64.
Click to expand...

It doesn't make sense for someone like Nintendo to try and compete with Nvidia and AMD on something they are an expert on by now. But eh, patents don't mean much anyway.
 
Marty-McFly

Marty-McFly

Banned
Coolwhhip said:
It doesn't make sense for someone like Nintendo to try and compete with Nvidia and AMD on something they are an expert on by now. But eh, patents don't mean much anyway.
Click to expand...
They could very likely still be working with Nvidia but putting their own spin on the tech (in fact, I think they are partnered contractually for the next decade). Let's not forget that the off the shelf Tegra Nvidia provided for the Switch was hacked in a matter of weeks. I'm sure Nintendo wants to take a more active approach in the design process especially considering how successful the Switch was conceptually. Refining the tech is now of much more importance than selling a new concept.
 
Last edited:
Marvel14

Marvel14

Banned
Marty-McFly said:
They are still likely working with Nvidia.

Saying that godfathers of gaming couldn't create well designed high tech hardware if they wanted to is ridiculous.

You must have missed the Gamecube and N64.
Click to expand...
You really are a Nintendo spokesperson aren't you?

Any company that hasn't been working at the cutting edge of complex electronics manufacture for decades would not be good enough at to go it alone.

Especially when their competitors have been pushing the envelope during all that time.

Nintendo have worked with IBM, Qualcomm and many other high tech companies to provide bespoke tech into their products for decades.

Saying Nintendo could do all of that by itself is what's really ridiculous. And only someone advertising for the Nintendo brand would be blinkered enough to say it.
 
Last edited:
Marty-McFly

Marty-McFly

Banned
Marvel14 said:
You really are a Nintendo spokesperson aren't you?

Any company that hasn't been working at the cutting edge of complex electronics manufacture for decades would not be good enough at to go it alone.

Especially when their competitors have been pushing the envelope during all that time.

Nintendo have worked with IBM, Qualcomm and many other high tech companies to provide bespoke tech into their products for decades.

Saying Nintendo could do all of that by itself is what's really ridiculous. And only someone advertising for the Nintendo brand would say it.
Click to expand...
Who said they are going at it alone? And the presumption that the most successful and established gaming company in history doesn't have the resources to be "good enough" to design their own tech is laugh inducing. No offence.
 
Dream-Knife

Dream-Knife

Banned
Those drawings show upresing a 540p image to 1080p (unless I'm misunderstanding). So, I don't think this is what people think it is.
 
Marty-McFly

Marty-McFly

Banned
Dream-Knife said:
Those drawings show upresing a 540p image to 1080p (unless I'm misunderstanding). So, I don't think this is what people think it is.
Click to expand...
Definitely misunderstanding. This is AI upscaling for those savvy enough to read through the proposed patents. Mentioning Tensor Cores is pretty much confirmation this is some DLSS 2.0 variant or something
 
Last edited:
Marty-McFly

Marty-McFly

Banned
Multiple possibilities. Note that we've known for a while that NERD actually does have their own machine learning engine, which was used to upscale the pre-rendered videos in Super Mario Sunshine.

1 Nintendo wants to have their own solution for future proofing, so that if they ever do split from Nvidia they will still be able to use ML upscaling
2 Nintendo built their own ML solution that plays better with their own internal engines
3 Nintendo built their own ML solution that plays better with older legacy content
4 Nintendo built their own ML solution that plays better with their graphical style than general DLSS does

The patents reference using Nvidia's tensor cores to implement the ML software solution, so it does seem like the infrastructure used will be Nvidia's, but they obviously want full control over the algorithm.
 
Last edited:
lils

lils

Member
it seems strange that they would develop their own when nvidia and amd have poured billions into this
 
StateofMajora

StateofMajora

Banned
Neat. Although it's probably dlss, maybe Nintendo are working with Nvidia to improve it for their custom tegra chip. It's worth pointing out that whatever kind of chip the switch 2 uses, it will need to be more efficient at machine learned image reconstruction due to inevitably far fewer tensor cores than even an rtx 2060.

Nintendo also may not like aspects of current dlss like sharpening or ghosting, or how it can occlude small particles i.e. Death stranding.

I just hope this thing comes out before metroid prime. I wanted bay 3 to be on switch 2 but I lost there lol.
 
Marty-McFly

Marty-McFly

Banned
lils said:
it seems strange that they would develop their own when nvidia and amd have poured billions into this
Click to expand...
As I mentioned the patents reference using Nvidia's tensor cores to implement the ML software solution, so it does seem like the infrastructure used will be Nvidia's, but they obviously want full control over the algorithm.
 
Last edited:
Kazza

Kazza

Member
Coolwhhip said:
Nintendo creating their own DLSS is like a monkey designing a space ship.

Why would they unless they drop nvidia?
Click to expand...

I kinda agree with this. If creating your own version of DLSS is so easy, why have AMD been struggling with it so much? (and it goes without saying that they have much more experience in GPU design than Nintendo)

It would be a typical Nintendo dick move to try and stab Nvidia in the back though :messenger_tears_of_joy:
 
Nautilus

Nautilus

Banned
This is obviously tech for Nintendo's 10th gen, assuming Nintendo actually go through with it.

Hard to say when thry will launch it, but if I had to guess, the earliest I can see it happening is 2024. If its earlier or later than that, I believe it will mostly depend on the Switch performance in 2022.
 
Soodanim

Soodanim

Member
It makes sense for Nintendo more than anyone else because they use modest hardware. The benefit will be greatest on their consoles. Nothing but a good thing. How good can a native 1080p image look with upscaling tech these days?
 
Woopah

Woopah

Member
It's definitely technology that can help Nintendo a lot. Upscaling could help with both battery life and third party support so will be interesting to see the results.

Question now is whether they position the next model as a revision or a successor.
 
Kenpachii

Kenpachii

Member
Could be they want to have it enabled on all devices even the current switch which nvidia doesn't wanna support with DLSS.
Could also be they going to ditch nvidia next round.
 
Rikkori

Rikkori

Member
That's very smart, because it allows them to maintain the technology required for the games in case they want to switch SOC vendors (i.e. AMD/InteL) in the future. Really the most expensive part for developing something like DLSS will be the hardware-time to do the training, but it's not a big deal at all. People way overrate how advanced something like DLSS is or what it does just because they hear "AI" and let their imagination run wild.
 
jigglet

jigglet

Banned
Ninty: we patented this tech
Industry: oh, we moved on 17 years ago
Dennis Dyack: for or against?
 
Last edited:
KungFucius

KungFucius

King Snowflake
ParaSeoul said:
what happened to gameplay over graphics?
Click to expand...
Isn't this really gameplay over graphics? I get that you are just trolling, but you missed the mark just like most trolls.

Develop low res game with good gameplay, upscale to higher res with hardware that is cheaper than the hardware to render without this algorithm. This prioritizes gameplay over graphics and cost of both development and HW.
 
yurinka

yurinka

Member
I gave a quick look at the patent and sounds like DLSS: cloud based, machine learning based image reconstruction to upscale resolution depending on the client's resolution (720p, 1080p, 4K, etc).

I don't know the details of Nvidia's DLSS, but I know it requires a specific GPU hardware and game specific implementation, which makes me think Nintendo would apply this in a totally different hardware where all games implement it (Switch's next gen successor, call it Switch 2 or Super Switch). What is the difference between this an Nvidia's solution? If it's the same, why Nintendo patents it instead of Nvidia? Didn't Nvidia already had it patented?

JLB said:
4k on a mobile size display is stupid.
Click to expand...
No, there are smartphones with 4K displays and they look great. Over time the resolution of phones and tablets increase and their price decrease. It's just a matter of time that they will become standard.

Dream-Knife said:
Those drawings show upresing a 540p image to 1080p (unless I'm misunderstanding). So, I don't think this is what people think it is.
Click to expand...
Yes, but in another place it mentions other resolutions like 4K.


Spukc said:
2022
Click to expand...
As I remember they said they expect Switch to last around 4 years, so I assume they'll release its successor 1 or 2 years before that. I'd say 2024 is more likely.
 
Last edited:
Buggy Loop

Buggy Loop

Gold Member
I really hope they let Nvidia handle it and that it’s just they want a proprietary solution like their API, but developed by Nvidia.

I can’t imagine Nintendo not fucking it up somehow, with terrible latencies or poor image quality if they were to attempt this.
 
jigglet

jigglet

Banned
Buggy Loop said:
I really hope they let Nvidia handle it and that it’s just they want a proprietary solution like their API, but developed by Nvidia.

I can’t imagine Nintendo not fucking it up somehow, with terrible latencies or poor image quality if they were to attempt this.
Click to expand...

Agreed, why reinvent this wheel, when your reinvention is only going to be worse.
 
Ev1L AuRoN

Ev1L AuRoN

Member
540p should reconstruct back to 1080p like Nvidia's DLSS balance. For a 4k output the base resolution must be at least 1080p native.
 
StateofMajora

StateofMajora

Banned
Ev1L AuRoN said:
540p should reconstruct back to 1080p like Nvidia's DLSS balance. For a 4k output the base resolution must be at least 1080p native.
Click to expand...
Ultra performance dlss is 720p to 4k.

But if Nintendo were to use dlss on their own software it probably wouldn't start so low, but it could vary from team to team.

This gen there's a wide gap in resolution Nintendo uses, i.e. Yoshi crafted world with UE4 and nice GI lighting is 540p while mk8 is 1080p so yeah lol.
 
Last edited:
It's all in the wrist

It's all in the wrist

Member
Anything that tries to be 4K, but not natively 4K, looks artificial to me. Nvidia's AI upscaling looks really bad (identical to cranking the sharpness up), and I don't like the artifacts (stippling, for lack of a better word) created by checkerboarding.

Not too hyped on this tech, but I'll take anything at this point over a Switch. Games are starting to look gross outside off Nintendo's stuff.
 
Last edited:
Kenpachii

Kenpachii

Member
In control DLSS 2.0

4k resolution u got 4 options

1) quality = 2560x1440
2) balance = 2227x1253
3) performance = 1920x1080
4) ultra performance = 1280x720
 
You must log in or register to reply here.

Similar threads

Nvidia Hints at DLSS 10 Delivering Full Neural Rendering, Potentially Replacing Rasterization and Ray Tracing
49
5K
SHA replied
Eurogamer "Avatar: Frontiers of Pandora is another PS5 Pro patch that arguably looks worse than the standard PS5 version. What's going on?"
2 3 4 5 6
276
13K
Bojji replied
Real-Time Neural Texture Upsampling in God of War Ragnarok on PS5
2
77
10K
Justin9mm replied
Sony Patents: Prevent Players From Accessing Age-Restricted Content, Cross-Platform Video Game NFTs, Method to Detect ADHD, Other Disorders
Business Game Dev
0
908
IbizaPocholo replied
EA Patent Uses Spatial Partitioning for Improved Graphics Rendering - Wants to Get Better at Predicting Players' Skill Levels
Game Dev
4
1K
Elysion replied
Top Bottom